Skip to content

MDBF-1185 msan/ubsan/asan with clang 22#912

Open
grooverdan wants to merge 3 commits into
MariaDB:devfrom
grooverdan:msan_ubsan_22
Open

MDBF-1185 msan/ubsan/asan with clang 22#912
grooverdan wants to merge 3 commits into
MariaDB:devfrom
grooverdan:msan_ubsan_22

Conversation

@grooverdan

@grooverdan grooverdan commented Feb 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Here we phase in clang 22 as a msan and ubasan builder.

If all goes well, we'll remove clang-20 out.

Add New Build template

Checklist

  • [N/A] Make changes os_info.yaml
  • Schedule the builder for the appropriate branch in constants.py
  • [Reused ] Add builder configuration
  • [N/A] Update workflows as appropriate i.e.: .github/workflows/build-X-based.yml
  • [N/A?] Update front page - master-web/templates/home.jade
  • [N/A - not a release platform] Add to MariaDB Documentation page for Platform Deprecation Policy

@RazvanLiviuVarzaru

RazvanLiviuVarzaru commented Feb 23, 2026

Copy link
Copy Markdown
Collaborator

Hi, @grooverdan !

My suggestion is to use sg-bbw1 as a worker machine.
@fauust can you please configure it to connect to master-migration?

This is the estimated load of this patch: https://github.com/MariaDB/buildbot/actions/runs/22293877010?pr=912

image

@grooverdan grooverdan force-pushed the msan_ubsan_22 branch 2 times, most recently from 063a9c2 to 2531155 Compare February 25, 2026 02:59
@grooverdan

grooverdan commented Feb 25, 2026

Copy link
Copy Markdown
Member Author

Arch: amd64

Worker Total Jobs Requested Jobs
hz-bbw6 96 20
hz-bbw8 110 110
hz-bbw9 110 110
bg-bbw5-x64 8 0
sg-bbw1 128 56

note llvm/clang 22 release hasn't happened. I think we'll container rebuild when that happens for a better dev test.

@RazvanLiviuVarzaru

RazvanLiviuVarzaru commented Feb 25, 2026

Copy link
Copy Markdown
Collaborator

Arch: amd64

Worker Total Jobs Requested Jobs
hz-bbw6 96 20
hz-bbw8 110 110
hz-bbw9 110 110
bg-bbw5-x64 8 0
sg-bbw1 128 56
note llvm/clang 22 release hasn't happened. I think we'll container rebuild when that happens for a better dev test.

Thanks, @grooverdan .
Can you tell me please when this should be scheduled for Production?
Because I can apply the changes in buildbot.dev.mariadb.org without actually merging and then you can use my server-fork to test as much as you like.

I am asking this because merging to DEV it is also a commitment to "soon will be released in buildbot production" and if this is not happening anytime soon it might get reverted if I want to do a release.

@grooverdan grooverdan mentioned this pull request Feb 26, 2026
Merged
@grooverdan

grooverdan commented Feb 26, 2026

Copy link
Copy Markdown
Member Author

https://discourse.llvm.org/t/llvm-22-1-0-released/89950

Released - hence #916

Can you tell me please when this should be scheduled for Production?
Because I can apply the changes in buildbot.dev.mariadb.org without actually merging and then you can use my server-fork to test as much as you like.

I've done the local #916 build and its passed its non-debug builds, server 10.11 on MSAN and ASAN/UBSAN. I'm not expecting massive failure.

Sure - lets take a non-merged test on DEV

If you've got time to trigger a 10.11, main, and if resources 11.4 (as a middle), I'll go through the results and report back here.

@grooverdan
MDBF-1185 msan/ubsan/asan with clang 22
ff553bf 
Here we phase in clang 22 as a msan and ubasan builder.

New builders get sg_bbw1 as a worker.

If all goes well, we'll remove clang-20 out.
@grooverdan grooverdan marked this pull request as draft April 2, 2026 07:08
@grooverdan grooverdan marked this pull request as ready for review April 3, 2026 06:53
@grooverdan grooverdan mentioned this pull request Apr 3, 2026
Merged
@grooverdan grooverdan force-pushed the msan_ubsan_22 branch 2 times, most recently from 2d00f1c to 912ac74 Compare May 27, 2026 07:58
@grooverdan
MDBF-1185 msan/ubsan/asan with clang 22 (experiments)
5d07cfb 
With the new clang 22 msan, ubasan builders there
is the opportunity to test new suites/options.

As new versions generally allow a little time to experiment with
different options, added an experimental option to helpers.

The spider suite is enabled in Debug but not RelWithDeb info
Debug is proved stable, so check if the non-debug is also.
This proved quick and stable locally.

On testing cursor and view protocols on msan and ubasan
in both debug and non-debug, the ubasan non-debug was
showing the same few trivial tests to correct as elsewhere
and ran significantly faster.

msan view showed a test failure that occurred without MSAN.
This was fast enough in a non-debug mode.
@fauust

fauust commented May 29, 2026

Copy link
Copy Markdown
Collaborator

Guys, sorry, missed that one, let me know if still relevant!

@RazvanLiviuVarzaru

RazvanLiviuVarzaru commented Jun 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

@fauust This is to configure the sg-bbw1 machine for Buildbot:
https://git.mariadb.org/MariaDB/sysadmin/pulls/71
If ok, please merge & deploy.

@grooverdan Since clang-22 builders are resource demanding, the plan would be:

  • have the builds run on the Quape machine for now
  • since the machine is idle at the moment, we can do as many dev tests as we like
  • have the builders deployed in production on the Quape machine. Since there's only one worker we can expect a fair amount of queues on busy days
  • once they are stable in production, ping me and I will handle removing clang-20 builders and assigning the clang-22 builders to a bigger worker pool (that can handle branch protection requirements for load and stability)

@fauust

fauust commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

@fauust This is to configure the sg-bbw1 machine for Buildbot: https://git.mariadb.org/MariaDB/sysadmin/pulls/71 If ok, please merge & deploy.

done

RazvanLiviuVarzaru
RazvanLiviuVarzaru reviewed Jun 2, 2026
View reviewed changes
Comment thread configuration/builders/sequences/sanitizers.py
@RazvanLiviuVarzaru

RazvanLiviuVarzaru commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

Hi, @grooverdan !

Here's the patch status on all branches, from the HEAD commits at the time of triggering.
I'm sorry for such a long delay in testing.
Moving forward, the builders should be green before merging this patch. I've re-assigned the MDBF to you.

Branch Commit Builder State
10.6 70a88a6b6e11d3cb840d5b27d05d967aff099ef0 amd64-msan-clang-22-debug ❌ failed compile (failure)
10.6 70a88a6b6e11d3cb840d5b27d05d967aff099ef0 amd64-ubasan-clang-22-debug ❌ failed compile (failure)
10.6 70a88a6b6e11d3cb840d5b27d05d967aff099ef0 amd64-msan-clang-22 ❌ failed MTR S3 (failure)
10.6 70a88a6b6e11d3cb840d5b27d05d967aff099ef0 amd64-ubasan-clang-22 ❌ failed MTR normal (failure) MTR S3 (failure)
10.11 936f04c4f9bd5d66014942bd27e1996b2e740626 amd64-msan-clang-22-debug ❌ failed compile (failure)
10.11 936f04c4f9bd5d66014942bd27e1996b2e740626 amd64-ubasan-clang-22-debug ❌ failed MTR normal (failure) MTR S3 (failure)
10.11 936f04c4f9bd5d66014942bd27e1996b2e740626 amd64-msan-clang-22 ❌ failed MTR S3 (failure)
10.11 936f04c4f9bd5d66014942bd27e1996b2e740626 amd64-ubasan-clang-22 ❌ failed MTR normal (failure) MTR S3 (failure)
11.4 d2ebea4febdbc102da09a3b51bea4ef2819a3843 amd64-msan-clang-22-debug ❌ failed compile (failure)
11.4 d2ebea4febdbc102da09a3b51bea4ef2819a3843 amd64-ubasan-clang-22-debug ❌ failed MTR normal (failure) MTR S3 (failure)
11.4 d2ebea4febdbc102da09a3b51bea4ef2819a3843 amd64-msan-clang-22 ❌ failed MTR S3 (failure)
11.4 d2ebea4febdbc102da09a3b51bea4ef2819a3843 amd64-ubasan-clang-22 ❌ failed MTR normal (failure) MTR S3 (failure)
11.8 080a7318acc2fc578b5915d1908f838b6f942de0 amd64-msan-clang-22-debug ❌ failed compile (failure)
11.8 080a7318acc2fc578b5915d1908f838b6f942de0 amd64-ubasan-clang-22-debug ❌ failed MTR normal (failure) MTR S3 (failure)
11.8 080a7318acc2fc578b5915d1908f838b6f942de0 amd64-msan-clang-22 ❌ failed MTR S3 (failure)
11.8 080a7318acc2fc578b5915d1908f838b6f942de0 amd64-ubasan-clang-22 ❌ failed MTR normal (failure) MTR S3 (failure)
12.3 8fd382adf6380ee299937b831ffdb4f22eaab37e amd64-msan-clang-22-debug ❌ failed compile (failure)
12.3 8fd382adf6380ee299937b831ffdb4f22eaab37e amd64-ubasan-clang-22-debug ❌ failed MTR normal (failure) MTR S3 (failure)
12.3 8fd382adf6380ee299937b831ffdb4f22eaab37e amd64-msan-clang-22 ❌ failed MTR S3 (failure)
12.3 8fd382adf6380ee299937b831ffdb4f22eaab37e amd64-ubasan-clang-22 ❌ failed MTR normal (failure) MTR S3 (failure) MTR cursor (failure)
13.0 09b9772eec85b72bca633f5885b4c4a029326214 amd64-msan-clang-22-debug ❌ failed compile (failure)
13.0 09b9772eec85b72bca633f5885b4c4a029326214 amd64-ubasan-clang-22-debug ❌ failed MTR normal (failure) MTR S3 (failure)
13.0 09b9772eec85b72bca633f5885b4c4a029326214 amd64-msan-clang-22 ❌ failed MTR S3 (failure)
13.0 09b9772eec85b72bca633f5885b4c4a029326214 amd64-ubasan-clang-22 ❌ failed MTR normal (failure) MTR S3 (failure) MTR cursor (failure) MTR view (failure)
main a06be0288f63f520aec95982cc6f872d26c4f23c amd64-msan-clang-22-debug ❌ failed compile (failure)
main a06be0288f63f520aec95982cc6f872d26c4f23c amd64-ubasan-clang-22-debug ❌ failed MTR normal (failure) MTR S3 (failure)
main a06be0288f63f520aec95982cc6f872d26c4f23c amd64-msan-clang-22 ❌ failed MTR S3 (failure)
main a06be0288f63f520aec95982cc6f872d26c4f23c amd64-ubasan-clang-22 ❌ failed MTR normal (failure) MTR S3 (failure) MTR cursor (failure) MTR view (failure)

@grooverdan
MDBF-1185 msan/ubsan/asan with clang 22 (fix)
9a073af 
asan_ubsan - mtr + reporter after view/cursor tests
@grooverdan

grooverdan commented Jun 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Thanks. Yikes.

10.6 was only intended to gain clang-22-msan per constants.

Bugs lodged and linked to MDBF-1185.

view/cursor bugs waiting on merge up.

ubasasn-debug - normal failures - waiting on mroonga merge up (10.11 only still) and connector ODBC update or code error supressions.

With the amd64-msan-clang-22-debug compile failure, can you trigger some 10.11+ builds with CMAKE_C{,XX}_FLAGS=-O1 to see what remains after the build step. I've verified this can get through the build. - never mind - crashes on bootstrap MDEV-38419

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@RazvanLiviuVarzaru RazvanLiviuVarzaru RazvanLiviuVarzaru left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@grooverdan @RazvanLiviuVarzaru @fauust