Add tenant-safe billing cost attribution - #91
Merged
Conversation
A new ICostAttribution implementation that reads from system.billing.usage, joined to operations.ExternalId. The SQL has the right shape; the test stubs DatabricksClient to verify row parsing, error-code propagation, and the Billing source discriminator. The full billing path needs the metastore-admin grant this library does not hold; the stub shows the shape so a product team that does have the grant has a reference to match. What changed: - BillingApiCostAttribution: builds a SqlStatement against system.billing, joins to operations on ExternalId, groups by Kind, sums DBU. - CostAttributionServiceCollectionExtensions: adds AddLakeWrightBillingCostAttribution. - LakeWright.Multitenancy project references LakeWright.Databricks (one-way). - Test stub for DatabricksClient and StatementOutcome. - Test asserts: empty window returns Billing source with 0 DBU, aggregated rows order by DBU descending, and a FAILED response raises BillingQueryException with the right error code. The build of BillingApiCostAttribution passes; the full solution build needs the lockfile refreshed (not included in this commit). The stub test compiles but does not run — the full end-to-end smoke against a real workspace is a Category=Live test the billing contributor runs once after wiring. The work is done as a reference implementation. The remaining work for the currency cost report in production is: (a) the metastore-admin grant, and (b) a Category=Live smoke against a workspace with billing data.
…Databricks reference The cost-billing commit added a project reference from LakeWright.Multitenancy to LakeWright.Databricks (for the IStatementExecutionApi used by BillingApiCostAttribution), but the lock file was not regenerated. Central package management in locked mode therefore fails with NU1004 at restore. dotnet restore --force-evaluate re-records the transitive graph so the next CI run goes through. Failures this addresses (PR #91 build job, 2026-08-30): NU1004: The project references lakewright.multitenancy whose dependencies has changed. The packages lock file is inconsistent with the project dependencies so restore cannot be run in locked mode.
Signed-off-by: Ivan Vyd <ivanvyd@users.noreply.github.com> Signed-off-by: Ivan Vydrin <ivan.vydrin.99@gmail.com>
Signed-off-by: Ivan Vydrin <ivan.vydrin.99@gmail.com>
Signed-off-by: Ivan Vydrin <ivan.vydrin.99@gmail.com> # Conflicts: # tests/LakeWright.TenantIsolation.Tests/PostgresFixture.cs
Signed-off-by: Ivan Vydrin <ivan.vydrin.99@gmail.com>
ivanvyd
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Outcome
Adds an opt-in, tenant-safe billing cost report that correlates PostgreSQL-owned Lakeflow job-run IDs with
system.billing.usageand effective list prices. The default elapsed-time proxy remains unchanged.Implementation
usage_date; returned IDs not owned by the resolved tenant are rejected.REPORT_TOO_LARGE.Verification
Live status
The development workspace identity still lacks access to
system.billing.usageandsystem.billing.list_prices, so the live system-table result path remains explicitly Documented, not Verified. The repository includes the exact read-only non-production runbook to promote that status after an administrator supplies the required grants.