Skip to content

validate.md

Latest commit

 

History

History
654 lines (450 loc) · 20.7 KB

validate.md

File metadata and controls

654 lines (450 loc) · 20.7 KB
title Validate
sidebar_position 4

Validate Feature

The Validate feature provides a comprehensive translation validation system designed to ensure translation completeness and quality before production deployment. It serves as a quality gate in CI/CD pipelines by checking the status of all translation resources across all configured locales and collections.

Overview

The validate command is a CLI-only, non-interactive tool that performs exhaustive validation of your entire translation inventory. Unlike other commands that may focus on subsets of data, validate checks everything and collects all validation results before reporting. This comprehensive approach ensures you have complete visibility into translation status before release.

Key Characteristics

  • CLI-only: No API or UI implementation
  • Non-interactive: Designed for CI/CD environments, never prompts for input
  • Comprehensive: Validates ALL resources across ALL locales and collections (locales can optionally be excluded with --skip-locales)
  • Non-short-circuiting: Collects ALL validation results before reporting (does not stop at first error)
  • Configurable strictness: Supports both strict (production) and relaxed (staging) validation modes
  • CI-friendly: Clear exit codes and structured output for pipeline integration

Usage

lingo-tracker validate [options]

Options

Option Description Default
--allow-translated Treat 'translated' status as warning instead of failure false
--skip-locales <locales> Comma-separated list of locales to exclude from validation None

Exit Codes

  • 0 - All validations passed (all resources verified or only warnings)
  • 1 - Validation failures found, OR configuration errors (config file missing or invalid JSON, no collections configured, no target locales configured, all target locales were skipped)

Validation Rules

The validate command categorizes resources based on their translation status:

Status Default Behavior With --allow-translated Description
new ❌ Failure ❌ Failure Resource not yet translated
stale ❌ Failure ❌ Failure Base value changed, translation out of sync
translated ❌ Failure ⚠️ Warning Has translation but not verified
verified ✅ Success ✅ Success Reviewed and approved
(missing) ❌ Failure ❌ Failure No status entry for this locale — treated as new

Validation Philosophy

  1. Strict by default: Production deployments should only include verified translations
  2. Configurable relaxation: Staging environments may accept translated (but unverified) content
  3. Never allow incomplete: New and stale translations always fail validation
  4. Comprehensive reporting: Show all issues, not just the first one encountered

How It Works

Validation Process

  1. Load Configuration: Read .lingo-tracker.json from project root
  2. Identify Scope: Determine all collections and all target locales (excludes base locale)
  3. Load Resources: Read all translation resources from all collections
  4. Validate Each Locale: For every target locale, check the status of every resource
  5. Collect Results: Accumulate all failures and warnings (no early exit)
  6. Generate Summary: Create comprehensive report grouped by locale
  7. Display Results: Show complete validation summary to console
  8. Exit: Return appropriate exit code based on overall validation status

What Gets Validated

  • Collections: ALL configured collections (no filtering)
  • Locales: ALL target locales by default (base locale always excluded; use --skip-locales to exclude additional locales)
  • Resources: EVERY resource in every collection
  • Statuses: Checked against validation rules for each locale

What Doesn't Get Validated

  • Base locale (source translations are authoritative by definition)
  • ICU message format syntax (use separate linting tools)
  • File structure or JSON validity (handled during resource loading)

Resources with no status entry for a locale are treated as new (failure), not skipped.

Skipping Locales

Use --skip-locales to exclude one or more locales from validation. This is useful when a locale has been added to the project configuration but its translations are still being prepared and are not yet ready for validation.

# Skip a single locale
lingo-tracker validate --skip-locales fr

# Skip multiple locales
lingo-tracker validate --skip-locales fr,de

Behaviour:

  • Unknown locales — locale codes not present in config.locales emit a console warning (⚠️ Skipping unknown locale '...' — not in configured locales) and are silently ignored.
  • Base locale — specifying the base locale in --skip-locales is silently ignored (the base locale is never validated regardless).
  • All target locales skipped — if every target locale is excluded, the command exits with code 1 rather than reporting a vacuous pass.
  • Validation summary — skipped locales appear in the summary between "Locales Validated" and "Collections Validated": Skipped Locales: fr, de (2).

CI/CD example — skip a locale in-progress while still gating production:

- name: Validate translations (skip in-progress locale)
  run: lingo-tracker validate --skip-locales de

Combine with --allow-translated for a staging-friendly gate that only validates the locales that are complete:

lingo-tracker validate --skip-locales de --allow-translated

Examples

Basic Usage

Strict validation (production):

lingo-tracker validate

Relaxed validation (staging):

lingo-tracker validate --allow-translated

Output Examples

All Verified (Success)

✅ Validation PASSED

📊 Validation Statistics:
──────────────────────────────────────────────────
  Total Resources Validated: 102
  Unique Resource Keys: 51
  Locales Validated: 2
  Collections Validated: 1

📈 Status Breakdown:
──────────────────────────────────────────────────
  ✅ Verified: 102
  ✏️  Translated: 0
  ⚠️  Stale: 0
  ❌ New: 0

──────────────────────────────────────────────────
📋 Summary:
  Total Failures: 0
  Total Successes: 102

✅ Validation passed successfully!

Exit code: 0

Mixed Status (Failure)

❌ Validation FAILED

📊 Validation Statistics:
──────────────────────────────────────────────────
  Total Resources Validated: 102
  Unique Resource Keys: 51
  Locales Validated: 2
  Collections Validated: 1

📈 Status Breakdown:
──────────────────────────────────────────────────
  ✅ Verified: 93
  ✏️  Translated: 3
  ⚠️  Stale: 2
  ❌ New: 4

❌ Failures (9):
──────────────────────────────────────────────────
  Locale: es (6 failures)
    ❌ [main] apps.common.buttons.submit (new)
    ❌ [main] apps.common.errors.network (new)
    ❌ [main] apps.features.dashboard.title (new)
    ⚠️ [main] apps.common.buttons.save (stale)
    ⚠️ [main] apps.features.settings.label (stale)
    ✏️ [main] apps.common.buttons.cancel (translated)

  Locale: fr-ca (3 failures)
    ❌ [main] apps.common.buttons.submit (new)
    ✏️ [main] apps.features.dashboard.subtitle (translated)
    ✏️ [main] apps.features.profile.header (translated)

──────────────────────────────────────────────────
📋 Summary:
  Total Failures: 9
  Total Successes: 93

❌ Validation failed. Please review the failures above.

Exit code: 1

With --allow-translated Flag (Warnings)

✅ Validation PASSED

📊 Validation Statistics:
──────────────────────────────────────────────────
  Total Resources Validated: 102
  Unique Resource Keys: 51
  Locales Validated: 2
  Collections Validated: 1

📈 Status Breakdown:
──────────────────────────────────────────────────
  ✅ Verified: 98
  ✏️  Translated: 4
  ⚠️  Stale: 0
  ❌ New: 0

⚠️  Warnings (4):
──────────────────────────────────────────────────
  Resources with "translated" status (not yet verified):
  Locale: es (3 warnings)
    ✏️  [main] apps.common.buttons.cancel (translated)
    ✏️  [main] apps.features.dashboard.subtitle (translated)
    ✏️  [main] apps.features.profile.header (translated)

  Locale: fr-ca (1 warnings)
    ✏️  [main] apps.common.buttons.apply (translated)

──────────────────────────────────────────────────
📋 Summary:
  Total Failures: 0
  Total Warnings: 4
  Total Successes: 98

✅ Validation passed with warnings.

Exit code: 0

CI/CD Integration

GitHub Actions

Production pipeline (strict):

name: Production Deploy

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '22'

      - name: Install dependencies
        run: pnpm install

      - name: Validate translations
        run: lingo-tracker validate

      - name: Build
        run: pnpm run build

      - name: Deploy
        run: pnpm run deploy:prod

Staging pipeline (relaxed):

name: Staging Deploy

on:
  push:
    branches: [develop]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version: '22'

      - name: Install dependencies
        run: pnpm install

      - name: Validate translations (allow translated)
        run: lingo-tracker validate --allow-translated

      - name: Build
        run: pnpm run build

      - name: Deploy
        run: pnpm run deploy:staging

Custom Script Integration

#!/bin/bash

echo "Running pre-deployment validation..."

# Validate translations
lingo-tracker validate

# Check exit code
if [ $? -ne 0 ]; then
    echo "❌ Translation validation failed. Deployment aborted."
    exit 1
fi

echo "✅ Translation validation passed. Proceeding with deployment..."

# Continue with build and deployment
pnpm run build
pnpm run deploy

Use Cases

1. Production Quality Gate

Scenario: Ensure only complete, verified translations reach production

Configuration: Default strict validation

lingo-tracker validate

Outcome: Deployment blocked if any translations are new, stale, or unverified

2. Staging Preview

Scenario: Deploy to staging with translated but unverified content for review

Configuration: Relaxed validation with --allow-translated

lingo-tracker validate --allow-translated

Outcome: Deployment allowed with warnings for unverified translations

3. Pull Request Checks

Scenario: Validate translation status in pull requests before merge

Configuration: Run as PR check with strict validation

name: PR Checks

on: [pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Validate translations
        run: lingo-tracker validate

Outcome: PR blocked if translations are incomplete

4. Translation Readiness Report

Scenario: Check translation completeness before sending to translators

Configuration: Run validate to identify gaps

lingo-tracker validate > translation-status.txt

Outcome: Comprehensive report of all translation gaps

5. Multi-Locale Deployment

Scenario: Deploy to production with different locale rollout strategies

Configuration: Use validation to verify each locale is ready

# Check if Spanish translations are production-ready
lingo-tracker validate

# If validation passes, deploy all locales
pnpm run deploy

Outcome: All locales validated before deployment

Best Practices

1. Use in CI/CD Pipelines

Always integrate validate into your CI/CD pipeline as a quality gate:

- name: Validate translations
  run: lingo-tracker validate

This prevents incomplete translations from reaching production.

2. Different Rules for Different Environments

Use strict validation for production and relaxed for staging:

Production (main branch):

lingo-tracker validate

Staging (develop branch):

lingo-tracker validate --allow-translated

3. Run Before Building

Validate translations before building your application:

lingo-tracker validate && pnpm run build

This prevents wasted build time if translations are incomplete.

4. Document Validation Requirements

In your project's README or CONTRIBUTING guide, document validation requirements:

## Translation Requirements

Before deploying to production:
- All translations must be in 'verified' status
- Run `lingo-tracker validate` to check status
- Translation validation is enforced in CI/CD pipeline

5. Review Validation Output

Don't just check exit codes—review the output to understand what needs attention:

lingo-tracker validate | tee validation-report.txt

This creates a record of validation results for review.

6. Combine with Export

Use validate to identify gaps, then export for translation:

# Check status
lingo-tracker validate

# Export gaps for translation
lingo-tracker export --format xliff --status new,stale

Technical Details

Implementation

The validate feature is implemented across two layers:

Core Library (libs/core/src/lib/validate/)

  • types.ts: Type definitions for validation options, results, and resource details
  • validate-resources.ts: Core validation logic and resource status checking
  • generate-validation-summary.ts: Summary generation and formatting

CLI Application (apps/cli/src/commands/validate.ts)

  • Command definition and option parsing
  • Configuration loading
  • Core library integration
  • Console output and exit code handling

Resource Loading

The validate command reuses the existing loadResourcesFromCollections utility from the core library, ensuring consistency with export/import operations.

Translation Status Semantics

Validate reads pre-computed status values from resource metadata — it does not re-evaluate checksums. The status for each resource/locale pair is determined during resource loading by loadResourcesFromCollections. Validate maps a missing status entry to new via a null-coalescing fallback.

The meaning of each status value:

  • new: Resource has been added but not yet translated (explicitly stored status)
  • stale: Base checksum changed since the translation was last updated (source drifted)
  • translated: Translation exists and matches the current base checksum, but not yet verified
  • verified: Translation exists and has been explicitly marked as verified

A missing metadata entry for a locale is coalesced to new at validation time (already covered by the (missing) row in the Validation Rules table above).

Performance

  • Fast: Validation only reads metadata, no heavy computation
  • Scalable: Handles projects with 10,000+ resources efficiently

Limitations

No Collection or Pattern Filtering

Validate always checks ALL collections. There is no option to:

  • Validate specific collections only
  • Filter by tags or patterns

Locales can be excluded with --skip-locales, but collection-level filtering is not supported.

Rationale: Validation is a pre-release gate and should be comprehensive. Partial validation creates risk of missing incomplete translations.

No Verbose Mode

Unlike export/import commands, validate does not have a --verbose flag. Output always includes:

  • Statistics (total resources, unique keys, locales, collections validated)
  • Status breakdown by count (verified, translated, stale, new)
  • Failures grouped by locale (with collection name and status)
  • Warnings grouped by locale (when --allow-translated is used)
  • Summary footer with total counts and final verdict

Verified resources are not listed individually.

Rationale: CI logs should be concise. Showing all verified resources adds noise.

Output Truncation

Within each locale group, at most 100 failures or warnings are shown. The cap is applied independently per locale — a project with 5 locales could show up to 500 failure lines. When the per-locale limit is exceeded, the output includes ... and N more failures (or ... and N more warnings for the warnings section) to indicate additional items. This prevents extremely long CI logs for large projects with many missing translations.

No JSON Output

Validate only outputs human-readable text. There is no --json flag for structured output.

Rationale: CI systems capture console output effectively. JSON output can be added in future if needed.

No Summary File

Unlike export/import, validate does not write a summary file to disk.

Rationale: CI systems capture stdout/stderr. No clear storage location for files.

Future Enhancements

Features explicitly not included but could be added:

  1. Verbose Mode: Show detailed per-resource status with --verbose flag
  2. Collection Filtering: Validate specific collections with a --collection flag
  3. JSON Output: Structured output for programmatic consumption with --format=json
  4. Summary File: Write markdown summary to disk with --output flag
  5. Custom Exit Codes: Different codes for different failure types
  6. Threshold Configuration: Allow X% translated with --threshold=95
  7. Webhook Notifications: Send results to Slack/Teams

These enhancements can be added based on user feedback without breaking existing functionality.

Troubleshooting

Validation Fails with Many New Resources

Problem: Validation shows many resources with 'new' status

Solution:

  1. Export new resources: lingo-tracker export --status new
  2. Send to translators
  3. Import translated content: lingo-tracker import
  4. Validate again: lingo-tracker validate

Validation Fails with Stale Resources

Problem: Resources marked as 'stale' after base value changes

Solution:

  1. Export stale resources: lingo-tracker export --status stale
  2. Update translations
  3. Import updated content: lingo-tracker import
  4. Validate again: lingo-tracker validate

Validation Passes but Translations Seem Wrong

Problem: All resources verified but translations contain errors

Solution: Validation only checks status, not content quality. Review translations manually or use additional QA processes.

Validation Fails in CI but Passes Locally

Problem: Inconsistent validation results between environments

Solution:

  1. Ensure .lingo-tracker.json is committed to version control
  2. Verify translation files are committed
  3. Check that all required files are included in CI checkout
  4. Confirm same version of lingo-tracker is used

Exit Code Always 0 Even with Failures

Problem: CI pipeline not detecting validation failures

Solution: Ensure you're checking the exit code correctly:

lingo-tracker validate
if [ $? -ne 0 ]; then
    echo "Validation failed"
    exit 1
fi

Related Commands

  • export: Export untranslated resources identified by validate
  • import: Import translations to resolve validation failures
  • normalize: Fix metadata issues that may affect validation
  • bundle: Generate deployment bundles after validation passes

Summary

The validate command is an essential tool for maintaining translation quality in production deployments. By enforcing comprehensive validation as part of your CI/CD pipeline, you ensure that:

  • No incomplete translations reach production
  • Translation status is transparent and actionable
  • Quality standards are consistently enforced
  • Deployment confidence is high

Use strict validation for production and relaxed validation for staging to balance quality requirements with development velocity.