Skip to content

Conversation

@ZPascal
Copy link

@ZPascal ZPascal commented Jan 26, 2026

Description

This PR introduces a comprehensive SQL datasource plugin for Perses, enabling users to query and visualize data from PostgreSQL, MySQL, and MariaDB databases. The plugin includes a full-featured Explorer mode similar to the Prometheus plugin, providing both table and graph views for SQL query results.

Key Features

SQL Datasource Plugin

  • Support for PostgreSQL, MySQL, and MariaDB
  • TLS/SSL connection support with certificate validation
  • Secret management integration for secure credential storage
  • Connection string validation and error handling
  • Database-specific driver selection

SQL Time Series Query Plugin

  • Rich query editor with SQL syntax highlighting
  • SQL macro support for dynamic time-based queries:
    • $__timeFilter(column) - Generates time range filter
    • $__timeFrom - Query start time
    • $__timeTo - Query end time
    • $__interval - Auto-calculated interval
    • $__interval_ms - Interval in milliseconds
  • Backend proxy integration for secure query execution
  • Full support for Perses variables in queries

SQL Explorer Plugin (New)

  • Prometheus-style explorer with dual-tab interface
  • Table View: Raw query results with column sorting
  • Graph View: Time series visualization using existing Perses panels
  • Multi-query editor with add/remove capabilities
  • Panel preview functionality
  • Seamless integration with dashboard workflow

Architecture

The plugin follows Perses plugin architecture best practices:

Frontend (TypeScript/React)

  • Plugin module using Module Federation for dynamic loading
  • CUE schema validation for datasource and query configurations
  • React components for the datasource editor, query editor, and explorer UI
  • Utility functions for SQL macro replacement and time handling

Backend Integration

  • Backend proxy handles actual database connections
  • Secure credential management via secrets API
  • Support for sslmode=disable for development environments
  • Detailed error messages for connection and query failures

Testing

Unit Tests (12 tests, all passing)

  • SQL macro replacement logic (replace-sql-builtin-variables.test.ts)
  • SQL client configuration (sql-client.test.ts)
  • Datasource component rendering (SQLDatasource.test.tsx)

Test Data

  • Docker Compose setup with PostgreSQL, MySQL, and MariaDB
  • Sample databases with realistic time-series data
  • Adminer UI for database inspection
  • Makefile commands for easy database management

Documentation

  • README.md: Comprehensive usage guide with examples
  • CUE Schemas: Schema definitions for validation
  • Code Comments: Apache License headers and JSDoc documentation

Screenshots

SQL Datasource Configuration

SQL Datasource Editor showing PostgreSQL configuration with TLS options

SQL Explorer - Table View

Explorer showing query results in table format

SQL Plugin View

SQL Plugin view

Dashboard with SQL Panel

Complete dashboard using SQL datasource with time series visualization

Checklist

  • Pull request has a descriptive title and context useful to a reviewer.
  • Pull request title follows the [FEATURE] <commit message> naming convention.
  • All commits have DCO signoffs.
  • Test the SQL plugin for MariaDB
  • Test the SQL plugin for MySQL

UI Changes

  • Changes that impact the UI include screenshots and/or screencasts of the relevant changes.
  • Code follows the UI guidelines.
  • E2E tests are stable and unlikely to be flaky.
    • Note: E2E tests not yet implemented. Unit tests cover business logic, and manual testing has been performed. E2E tests can be added in a follow-up PR if required.

Additional Notes

What's Included

Production-ready features:

  • 3 database drivers (PostgreSQL, MySQL, MariaDB)
  • Full TLS/SSL support
  • SQL macro system
  • Explorer mode with table and graph views
  • Secret management integration
  • Comprehensive documentation
  • Unit test coverage for core logic

Future Enhancements (Not Blocking)

  • E2E tests for UI workflows
  • Additional UI component tests
  • ESLint configuration (TypeScript already catches most issues)
  • Performance optimizations for large datasets

Testing Instructions

  1. Start test databases:

    cd plugins/sql
    make db-up
  2. Run unit tests:

    npm test
  3. Build the plugin:

    npm run build
  4. Manual testing:

    • Configure a SQL datasource in Perses
    • Use the query editor to write SQL queries with macros
    • Open the Explorer to build queries interactively
    • Create a dashboard panel using the SQL datasource

Database Compatibility

Database Version Status
PostgreSQL 15+ ✅ Fully supported
MySQL 8.0+ ✅ Fully supported
MariaDB 10.6+ ✅ Fully supported

Migration Notes

This is a new plugin and does not require migration from existing configurations.

Related issues

Add PostgreSQL plugin support

Related PRs

Adapt the proxy output
Handle the MariaDB support for the backend proxy

Signed-off-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
@ZPascal ZPascal force-pushed the add-sql-plugin-support branch from e0cd822 to 82e7d52 Compare January 26, 2026 05:49
@ZPascal ZPascal marked this pull request as ready for review January 26, 2026 05:50
@ZPascal ZPascal requested review from a team and AntoineThebaud as code owners January 26, 2026 05:50
@ZPascal ZPascal requested review from Gladorme and removed request for a team January 26, 2026 05:50
@jgbernalp
Copy link
Contributor

Related to: perses/perses#2930

kind: #kind
spec: {
proxy: {
kind: "sqlproxy"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A big thanks for this huge work to cover a gap in the Perses plugin !

I have quickly looked at the code and actually this can be simplified as the definition of the sql proxy is already defined here: https://github.com/perses/shared/blob/main/cue/common/proxy/sql.cue

So it should give something like that:

kind: #kind
spec: {
  proxy: commonProxy.#SQLProxy
}

Copy link
Contributor

@rickardsjp rickardsjp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. Please add the sql plugin to the top level (monorepo root) package.json workspaces list. Otherwise, it will not be included in e.g. the build and lint commands. Once you include it, please take a look at the linter issues, too.

Comment on lines +1 to +29
export default {
displayName: 'sql',
preset: 'ts-jest',
testEnvironment: 'jsdom',
setupFilesAfterEnv: ['<rootDir>/src/setup-tests.ts'],
moduleNameMapper: {
'\\.(css|less|scss|sass)$': 'identity-obj-proxy',
'^echarts$': '<rootDir>/__mocks__/echarts.ts',
'^zrender$': '<rootDir>/__mocks__/zrender.ts',
},
transform: {
'^.+\\.tsx?$': ['ts-jest', {
tsconfig: {
jsx: 'react',
esModuleInterop: true,
},
}],
},
testMatch: [
'**/__tests__/**/*.ts?(x)',
'**/?(*.)+(spec|test).ts?(x)',
],
collectCoverageFrom: [
'src/**/*.{ts,tsx}',
'!src/**/*.d.ts',
'!src/**/index.ts',
'!src/setup-tests.ts',
],
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the shared jest config from ../jest.shared.ts like the other plugins.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants