Skip to content

Reduce Boost dependency footprint #3405

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
andrjohns wants to merge 15 commits into
base: develop
Choose a base branch
from
Open

Reduce Boost dependency footprint #3405

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
490411a
Replace boost random with std
andrjohns Aug 12, 2026
51eded8
Remove dead includes
andrjohns Aug 12, 2026
5178da7
Remove boost/algorithm usages
andrjohns Aug 12, 2026
ea93d96
Update RNG usage and test expectations
andrjohns Aug 12, 2026
a81db20
Replace boost/regex usage
andrjohns Aug 12, 2026
d50a260
Replace circular_buffer
andrjohns Aug 12, 2026
7c0ff41
Replace accumulators usage
andrjohns Aug 12, 2026
bbff16b
[Jenkins] auto-formatting by clang-format version 10.0.0-4ubuntu1
stan-buildbot Aug 12, 2026
0d91b3c
Update hardcoded expectations
andrjohns Aug 12, 2026
1764ef9
Update tolerance
andrjohns Aug 12, 2026
7192fd7
ring_buffer: guard 0-capacity, use moves, add tests
andrjohns Aug 15, 2026
e978161
Optimise/simplify string utils, add tests
andrjohns Aug 15, 2026
1e1c33e
Merge commit 'f7437d911374df788a9f108297bdb493b2ae5bd7' into HEAD
yashikno Aug 15, 2026
32814ac
[Jenkins] auto-formatting by clang-format version 10.0.0-4ubuntu1
stan-buildbot Aug 15, 2026
7d27504
Remove usages in tests
andrjohns Aug 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 2 additions & 12 deletions src/stan/analyze/mcmc/autocovariance.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@
#define STAN_ANALYZE_MCMC_AUTOCOVARIANCE_HPP

#include <stan/math/prim.hpp>
#include <boost/accumulators/accumulators.hpp>
#include <boost/accumulators/statistics/stats.hpp>
#include <boost/accumulators/statistics/variance.hpp>
#include <unsupported/Eigen/FFT>
#include <complex>
#include <vector>
Expand Down Expand Up @@ -87,16 +84,9 @@ void autocovariance(const Eigen::MatrixBase<DerivedA>& y,
Eigen::FFT<T> fft;
autocorrelation(y, acov, fft);

using boost::accumulators::accumulator_set;
using boost::accumulators::stats;
using boost::accumulators::tag::variance;
double variance = (y.array() - y.mean()).matrix().squaredNorm() / y.size();

accumulator_set<double, stats<variance>> acc;
for (int n = 0; n < y.size(); ++n) {
acc(y(n));
}

acov = acov.array() * boost::accumulators::variance(acc);
Comment on lines +87 to -99

@SteveBronder SteveBronder Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use a little variance function like the below that uses welford's algorithm so we can be a bit more precise

https://godbolt.org/z/Porcq1z8W

@andrjohns andrjohns Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting! I was looking into the speed/accuracy tradeoff from switching from vectorised Eigen to welford's, and it's showing the Eigen vectorised impl as being more accurate still: https://godbolt.org/z/G33637P6r.

I've also attached the version that did a timing comparison across a larger range as well.

I'm not overly-confident that Claude isn't missing something obvious though, are there any particular edge-cases or examples where you've seen the welford benefits that I can test with? I was always under the impression that the main motivator was for online/streaming cases rather than with a fully-materialised case.

variance_accuracy.cpp

@andrjohns andrjohns Aug 15, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do have existing welford estimators for variance and covariance in the Math library that I could switch to here and for the covariance below:

@roualdes roualdes Aug 15, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relative to @andrjohns' file variance_accuracy.cpp.

  • eigen_twopass is generally more accurate, but can fail in extreme cases. John Cook has a post about this. It's not clear how common such cases are.

  • I've always written the variance calculations for welford_seq as
    m2 += -m2 * inv + delta * delta * inv * (1 - inv); and then return m2 * n / (n - 1);, which proves a bit more accurate than welford_seq, and otherwise follows the pattern of the bullet point above.

acov = acov.array() * variance;
}

/**
Expand Down
31 changes: 9 additions & 22 deletions src/stan/analyze/mcmc/compute_potential_scale_reduction.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,6 @@
#include <stan/math/prim.hpp>
#include <stan/analyze/mcmc/autocovariance.hpp>
#include <stan/analyze/mcmc/split_chains.hpp>
#include <boost/accumulators/accumulators.hpp>
#include <boost/accumulators/statistics/stats.hpp>
#include <boost/accumulators/statistics/mean.hpp>
#include <boost/accumulators/statistics/variance.hpp>
#include <boost/math/distributions/normal.hpp>
#include <algorithm>
#include <cmath>
#include <vector>
Expand Down Expand Up @@ -74,30 +69,22 @@ inline double compute_potential_scale_reduction(
}
}

using boost::accumulators::accumulator_set;
using boost::accumulators::stats;
using boost::accumulators::tag::mean;
using boost::accumulators::tag::variance;

Eigen::VectorXd chain_mean(num_chains);
accumulator_set<double, stats<variance>> acc_chain_mean;
Eigen::VectorXd chain_var(num_chains);
double unbiased_var_scale = num_draws / (num_draws - 1.0);

for (int chain = 0; chain < num_chains; ++chain) {
accumulator_set<double, stats<mean, variance>> acc_draw;
for (int n = 0; n < num_draws; ++n) {
acc_draw(draws[chain][n]);
}
Eigen::Map<const Eigen::Matrix<double, Eigen::Dynamic, 1>> draw(
draws[chain], num_draws);

chain_mean(chain) = boost::accumulators::mean(acc_draw);
acc_chain_mean(chain_mean(chain));
chain_var(chain)
= boost::accumulators::variance(acc_draw) * unbiased_var_scale;
chain_mean(chain) = draw.mean();
chain_var(chain) = (draw.array() - chain_mean(chain)).matrix().squaredNorm()
/ (num_draws - 1.0);
}

double var_between = num_draws * boost::accumulators::variance(acc_chain_mean)
* num_chains / (num_chains - 1);
double var_between
= num_draws
* (chain_mean.array() - chain_mean.mean()).matrix().squaredNorm()
/ (num_chains - 1.0);
double var_within = chain_var.mean();

return sqrt((var_between / var_within + num_draws - 1) / num_draws);
Expand Down
28 changes: 14 additions & 14 deletions src/stan/io/json/json_data_handler.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@
#include <stan/io/json/json_error.hpp>
#include <stan/io/json/json_handler.hpp>
#include <stan/io/json/rapidjson_parser.hpp>
#include <stan/io/string_utils.hpp>
#include <stan/io/var_context.hpp>
#include <algorithm>
#include <cctype>
#include <iostream>
#include <ostream>
Expand All @@ -15,8 +17,6 @@
#include <string>
#include <utility>
#include <vector>
#include <boost/algorithm/string.hpp>
#include <boost/regex.hpp>

namespace stan {

Expand Down Expand Up @@ -146,9 +146,7 @@ class json_data_handler : public stan::json::json_handler {
array_start_r = 0;
}

inline std::string key_str() {
return boost::algorithm::join(key_stack, ".");
}
inline std::string key_str() { return stan::io::join(key_stack, "."); }

std::string outer_key_str() {
std::string result;
Expand All @@ -172,16 +170,19 @@ class json_data_handler : public stan::json::json_handler {
* and contain only letters, numbers, or an underscore.
*/
bool valid_varname(const std::string& name) {
static const boost::regex re("[a-zA-Z][a-zA-Z0-9_]*");
return boost::regex_match(name, re);
if (name.empty() || !std::isalpha(static_cast<unsigned char>(name[0])))
return false;
return std::all_of(name.begin() + 1, name.end(), [](unsigned char c) {
return std::isalnum(c) || c == '_';
});
}

bool is_array_tuples(const std::vector<std::string>& keys) {
std::vector<std::string> stack(keys);
std::string key;
stack.pop_back();
while (!stack.empty()) {
key = boost::algorithm::join(stack, ".");
key = stan::io::join(stack, ".");
if (slot_types_map[key] == meta_type::ARRAY_OF_TUPLES)
return true;
stack.pop_back();
Expand All @@ -194,12 +195,12 @@ class json_data_handler : public stan::json::json_handler {
std::string key;
stack.pop_back();
while (!stack.empty()) {
key = boost::algorithm::join(stack, ".");
key = stan::io::join(stack, ".");
if (slot_dims_map.count(key) == 1)
return slot_dims_map[key];
stack.pop_back();
}
key = boost::algorithm::join(keys, ".");
key = stan::io::join(keys, ".");
if (slot_dims_map.count(key) != 1)
unexpected_error(key, "not an array");
return slot_dims_map[key];
Expand All @@ -210,13 +211,13 @@ class json_data_handler : public stan::json::json_handler {
std::string key;
stack.pop_back();
while (!stack.empty()) {
key = boost::algorithm::join(stack, ".");
key = stan::io::join(stack, ".");
if (slot_dims_map.count(key) == 1)
break;
stack.pop_back();
}
if (stack.empty()) {
key = boost::algorithm::join(key_stack, ".");
key = stan::io::join(key_stack, ".");
unexpected_error(key, "ill-formed array");
}
slot_dims_map[key] = update;
Expand Down Expand Up @@ -354,8 +355,7 @@ class json_data_handler : public stan::json::json_handler {
continue;
}
std::vector<size_t> all_dims;
std::vector<std::string> slots;
split(slots, var.first, boost::is_any_of("."), boost::token_compress_on);
std::vector<std::string> slots = stan::io::split(var.first, ".", true);
std::string slot;
for (size_t i = 0; i < slots.size(); ++i) {
slot.append(slots[i]);
Expand Down
5 changes: 2 additions & 3 deletions src/stan/io/random_var_context.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@

#include <stan/io/var_context.hpp>
#include <stan/io/validate_dims.hpp>
#include <boost/random/uniform_real_distribution.hpp>
#include <algorithm>
#include <limits>
#include <random>
#include <string>
#include <vector>

Expand Down Expand Up @@ -50,8 +50,7 @@ class random_var_context : public var_context {
for (size_t n = 0; n < num_unconstrained_; ++n)
unconstrained_params_[n] = 0.0;
} else {
boost::random::uniform_real_distribution<double> unif(-init_radius,
init_radius);
std::uniform_real_distribution<double> unif(-init_radius, init_radius);
for (size_t n = 0; n < num_unconstrained_; ++n)
unconstrained_params_[n] = unif(rng);
}
Expand Down
28 changes: 14 additions & 14 deletions src/stan/io/stan_csv_reader.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
#ifndef STAN_IO_STAN_CSV_READER_HPP
#define STAN_IO_STAN_CSV_READER_HPP

#include <boost/algorithm/string.hpp>
#include <stan/io/string_utils.hpp>
#include <stan/math/prim.hpp>
#include <algorithm>
#include <cctype>
#include <istream>
#include <iostream>
Expand All @@ -15,8 +16,7 @@ namespace io {

inline void prettify_stan_csv_name(std::string& variable) {
if (variable.find_first_of(":.") != std::string::npos) {
std::vector<std::string> parts;
boost::split(parts, variable, boost::is_any_of(":"));
std::vector<std::string> parts = split(variable, ":");
for (auto& part : parts) {
int pos = part.find('.');
if (pos > 0) {
Expand All @@ -25,7 +25,7 @@ inline void prettify_stan_csv_name(std::string& variable) {
part += "]";
}
}
variable = boost::algorithm::join(parts, ".");
variable = join(parts, ".");
}
}

Expand Down Expand Up @@ -126,10 +126,10 @@ class stan_csv_reader {
size_t equal = lhs.find("=");
if (equal != std::string::npos) {
name = lhs.substr(0, equal);
boost::trim(name);
trim(name);
value = lhs.substr(equal + 1, lhs.size());
boost::trim(value);
boost::replace_first(value, " (Default)", "");
trim(value);
replace_first(value, " (Default)", "");
} else {
if (lhs.compare(" data") == 0) {
ss >> comment;
Expand All @@ -138,9 +138,9 @@ class stan_csv_reader {
size_t equal = lhs.find("=");
if (equal != std::string::npos) {
name = lhs.substr(0, equal);
boost::trim(name);
trim(name);
value = lhs.substr(equal + 2, lhs.size());
boost::replace_first(value, " (Default)", "");
replace_first(value, " (Default)", "");
}

if (name.compare("file") == 0)
Expand Down Expand Up @@ -176,7 +176,7 @@ class stan_csv_reader {
std::stringstream(value) >> metadata.chain_id;
} else if (name.compare("init") == 0) {
metadata.init = value;
boost::trim(metadata.init);
trim(metadata.init);
} else if (name.compare("seed") == 0) {
std::stringstream(value) >> metadata.seed;
metadata.random_seed = false;
Expand Down Expand Up @@ -209,7 +209,7 @@ class stan_csv_reader {
while (ss.good()) {
std::string token;
std::getline(ss, token, ',');
boost::trim(token);
trim(token);

if (prettify_name) {
prettify_stan_csv_name(token);
Expand Down Expand Up @@ -239,7 +239,7 @@ class stan_csv_reader {

// parse stepsize
std::getline(ss, line, '='); // stepsize
boost::trim(line);
trim(line);
ss >> adaptation.step_size;
if (lines == 2) // ADVI reports stepsize, no metric
return;
Expand All @@ -265,7 +265,7 @@ class stan_csv_reader {
for (int col = 0; col < cols; col++) {
std::string token;
std::getline(line_ss, token, ',');
boost::trim(token);
trim(token);
std::stringstream(token) >> adaptation.metric(row, col);
}
std::getline(ss, line);
Expand Down Expand Up @@ -335,7 +335,7 @@ class stan_csv_reader {
std::stringstream ls(line);
for (int col = 0; col < cols; col++) {
std::getline(ls, line, ',');
boost::trim(line);
trim(line);
try {
samples(row, col) = static_cast<double>(std::stold(line));
// If the value read is out of the range of representable values by
Expand Down
Loading
Loading