feat: add initial fuzz testing#271
Draft
oleonardolima wants to merge 15 commits intobitcoindevkit:masterfrom
Draft
feat: add initial fuzz testing#271oleonardolima wants to merge 15 commits intobitcoindevkit:masterfrom
oleonardolima wants to merge 15 commits intobitcoindevkit:masterfrom
Conversation
Pull Request Test Coverage Report for Build 16808127177Details
💛 - Coveralls |
oleonardolima
force-pushed
the
feat/add-initial-fuzz-testing
branch
2 times, most recently
from
31f422f to
2b885dc
Compare
Member
|
Could you include the rationale for having a separate crate for fuzzing? Is it because we need to use nightly cargo? |
|
Rather than requiring use of |
oleonardolima
force-pushed
the
feat/add-initial-fuzz-testing
branch
3 times, most recently
from
1166295 to
8422bc6
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
oleonardolima
force-pushed
the
feat/add-initial-fuzz-testing
branch
from
8422bc6 to
f309e1d
Compare
oleonardolima
force-pushed
the
feat/add-initial-fuzz-testing
branch
from
f309e1d to
3b70fa6
Compare
oleonardolima
force-pushed
the
feat/add-initial-fuzz-testing
branch
from
47ecc5f to
7f1fa74
Compare
|
Hello @oleonardolima thanks for this initial work. I'd like to continue this or work on it with you if you're fine with that. I left you some discord messages too. I'm adding the link to the report done by Wizardzine in case anyone else wants to read it to help quick reference because the report wasn't that much easy to find. https://gist.github.com/darosior/4aeb9512d7f1ac7666abc317d6f9453b |
Contributor
Author
Thanks, any help with adding other fuzz targets and enhancements are appreciated, though I need to do some cleanup on this PR first, but feel free to send your suggestions on top of this one on the fork. |
- creates a new `fuzz` crate, it's meant to run fuzz testing over bdk_wallet targets, with `cargo fuzz` (libFuzzer). - creates an initial `wallet_update` fuzz target for `bdk_wallet`. - creates an initial `fuzzed_data_provider` and `fuzz_utils` files with useful methods to consume the fuzzed data into `bdk_wallet` API-specific types. test(fuzzing): improve `bdk_wallet` fuzz target - renames the fuzz target to `bdk_wallet`. - add the `WalletAction` enum, in order to fuzz test different behaviors: wallet update, persistance/load, and tx creation. - use macros (e.g `try_consume_*`) in `fuzzed_data_provider` and `fuzz_utils` in order to properly handle an exhausted fuzzer byte stream, returning early. - update `Wallet::ApplyUpdate` target to use the newly added macros.
- update `bdk_wallet_fuzz` to use `rusqlite` feature. - update the created wallet in `bdk_wallet` fuzz target to use an in-memory sqlite database connection, initializing wallet with persistance. - add the `PersistAndLoad` scenario to `bdk_wallet` fuzz target.
- add the `CreateTx` scenario to `bdk_wallet` fuzz target. - add two new macros: `try_consume_tx_builder` and `try_consume_sign_options`, in order to build the specific structures and types required for tx creation, signing and applying to wallet.
oleonardolima
force-pushed
the
feat/add-initial-fuzz-testing
branch
from
7f1fa74 to
a6a44d3
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #271 +/- ##
=======================================
Coverage 86.67% 86.67%
=======================================
Files 25 25
Lines 8487 8487
=======================================
Hits 7356 7356
Misses 1131 1131
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
- Move wallet code to wallet/ subdirectory - Create workspace root Cargo.toml with wallet and fuzz members - Update fuzz crate to use path dependency for bdk_wallet - Maintain all existing functionality while improving project organization This restructure enables better isolation of the fuzz testing infrastructure and makes the project more modular. Co-Authored-By: Claude <noreply@anthropic.com>
- Add arbitrary crate dependency with derive feature - Create arbitrary_types module with basic wrapper types: - FuzzedTxid for transaction IDs - FuzzedBlockHash for block hashes - FuzzedOutPoint for transaction output references - FuzzedAmount with reasonable constraints (weighted towards smaller values) - FuzzedScript with size limits - Implement conversion methods to BDK wallet types This is the foundation for migrating from macro-based fuzzing to structure-aware fuzzing using the Arbitrary trait. Co-Authored-By: Claude <noreply@anthropic.com>
- Add FuzzedWalletAction enum with ApplyUpdate, CreateTx, and PersistAndLoad - Implement FuzzedSignOptions with all signing configuration fields - Add FuzzedTapLeavesOptions for taproot signing control - Create FuzzedTxBuilderOptions with transaction builder parameters: - Fee configuration (rate vs absolute) - Transaction options (locktime, version, ordering) - UTXO selection options - Output configuration options - Use weighted probabilities for more realistic option generation These types provide structured inputs for wallet operations instead of raw byte manipulation. Co-Authored-By: Claude <noreply@anthropic.com>
- Add FuzzedTxInput and FuzzedTxOutput with conversion methods - Implement FuzzedTransaction with wallet-aware output generation - Create FuzzedAnchor for confirmation data - Add FuzzedTxUpdate with all transaction update components - Implement FuzzedCheckpoint for chain progression - Create FuzzedUpdate combining all wallet update elements - Include conversion methods that integrate with wallet state - Use reasonable limits and weighted probabilities for data generation These types enable structured generation of blockchain data for fuzzing instead of manual byte manipulation. Co-Authored-By: Claude <noreply@anthropic.com>
- Add FuzzedRecipient for transaction outputs - Create FuzzedTxBuilder with comprehensive configuration: - Recipients list with amounts and scripts - Fee configuration (rate or absolute) - UTXO selection (manual add/unspendable marking) - Transaction options (locktime, version, ordering, sighash) - Change spending policies - PSBT options (witness UTXOs, xpubs, scripts) - Drain wallet and dust handling - Implement build_with_wallet() method that: - Supports both normal transactions and fee bumps - Integrates with wallet addresses for better coverage - Handles all builder configuration options - Pre-collects data to avoid borrow conflicts This provides a complete abstraction for transaction building during fuzzing. Co-Authored-By: Claude <noreply@anthropic.com>
- Add FuzzedWalletOperation enum combining all wallet operations: - ApplyUpdate with full update data - CreateTransaction with builder, sign options, and finalization - PersistAndLoad for state verification - Create FuzzInput struct as the main entry point with operation sequence - Implement execute() method that: - Processes all operations sequentially - Handles PersistedWallet instead of raw Wallet - Signs and finalizes transactions when configured - Verifies wallet state consistency - Gracefully handles expected errors - Export descriptor and network constants for reuse - Use weighted probabilities for operation selection This provides a complete execution engine for structured fuzzing inputs. Co-Authored-By: Claude <noreply@anthropic.com>
- Replace macro-based byte parsing with structured FuzzInput - Use the arbitrary trait for automatic input generation - Simplify target to just create wallet and execute operations - Remove all manual data parsing and macro invocations - Let the FuzzInput::execute() method handle all operation logic The fuzz target is now clean, simple, and type-safe, delegating all complexity to the structured arbitrary types. Co-Authored-By: Claude <noreply@anthropic.com>
- Remove fuzzed_data_provider.rs with all macro definitions - Remove fuzz_utils.rs with macro-based helper functions - Update lib.rs to only export the new arbitrary_types module - Add comprehensive documentation to lib.rs - Update README with detailed information about: - Architecture and benefits of structure-aware fuzzing - How to run and add new fuzz targets - How to reproduce crashes - Key components and their purpose This completes the migration from macro-based fuzzing to structure-aware fuzzing using the Arbitrary trait, resulting in cleaner, more maintainable, and more effective fuzz testing. Co-Authored-By: Claude <noreply@anthropic.com>
Optimizations implemented: - Limit operations to 1-5 per run (was unlimited) - Reduce transaction sizes to 1-3 inputs, 1-2 outputs (was 10 each) - Use simple fixed scripts or wallet addresses (was complex arbitrary) - Simplify updates with flags instead of full data structures - Remove expensive PersistAndLoad operation - Streamline transaction builder to essential options only Performance improvements: - ~15-20% increase in executions per second (142-150 vs 125) - Lower memory usage - Faster coverage growth - Better bug-finding efficiency The optimized fuzzer demonstrates that domain-specific optimizations can significantly improve fuzzing effectiveness while maintaining comprehensive testing coverage. Co-Authored-By: Claude <noreply@anthropic.com>
This reverts commit 225c1af.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
It's an initial work towards the goal of #61, it adds a new
fuzzcrate to the project which is meant to be used to add new fuzz target in order to increase the fuzz coverage.It's built on top of and with inspiration from previous Antoine's work through the most recent BDK Audit.
So far, this PR:
fuzzcrate withcargo fuzzrelying on LibFuzzer engine.UpdatetoWallet.fuzzed_data_providerandfuzz_utilswith common used fns to build BDK's-specific types from fuzzed bytes.Still to be done by this PR:
Notes to the reviewers
Are there any other BDK usage scenarios you'd like to see covered by a fuzz target ? Let's discuss it either on the issue or under this PR comments.
Changelog notice
TBD
Checklists
All Submissions:
just pbefore pushing