Report per operation timings in the multi editor and perspective tests - #4232
Merged
vogella merged 1 commit intoAug 14, 2026
Merged
Conversation
Contributor
Test Results 858 files 858 suites 50m 21s ⏱️ Results for commit 07fd8a9. ♻️ This comment has been updated with latest results. |
Contributor
Author
|
Change in test code, performance tests are not part of the CI verification (only compiled) |
Migrates PerspectiveSwitchTest the same way as OpenCloseEditorTest, EditorSwitchTest and OpenMultipleEditorTest: drop org.eclipse.test.performance, warm up before measuring, time the individual operations and print the distribution through UIPerformanceTestUtil.reportTimings. The performance database it reported to has not been configured for years, so commitMeasurements and assertPerformance were no-ops and the test could neither fail nor report anything. Each switch direction is now reported separately, since entering the two perspectives does not cost the same. When a perspective is missing the test skips through an assumption instead of printing to stdout and returning, which reported a pass for something never measured. Three of its four cases reference JDT perspectives and are skipped in this target platform, which is now visible in the test results. OpenMultipleEditorTest opens 90 instead of 100 editors: the workbench recycles the oldest editor once REUSE_EDITORS are open, which defaults to 99, so the hundredth open silently reused the first editor and the test measured a reuse rather than an open. An assertion now pins that down.
vogella
force-pushed
the
perf-tests-multi-editor-perspective
branch
from
August 13, 2026 09:04
2e1e66d to
07fd8a9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continues #4230 for the two tests that cover the operations users feel most: opening many editors and switching perspectives. Both drop org.eclipse.test.performance, warm up before measuring, time the individual operations and print the distribution, so a change in one operation can be attributed instead of disappearing into a single number that nothing recorded.
OpenMultipleEditorTest also reports the first and the last ten editors separately, which turns it into the only test we have that shows how cost scales with the number of open editors. Opening the 90th editor currently costs 20 to 67 percent more than the 10th, and closing an editor while 90 are open costs about three times as much as when 10 remain. Because both ends come from the same run, that comparison is unaffected by machine load, which the absolute numbers very much are.
While adding assertions I found that the test never measured what it claimed: the workbench recycles the oldest editor once REUSE_EDITORS are open, defaulting to 99, so the hundredth open silently reused the first editor. The count is now 90 with an assertion explaining the threshold. PerspectiveSwitchTest had a related problem, silently returning and printing to stdout when a perspective is missing, which reported a pass for something never measured; it now skips through an assumption, so the three JDT cases show up as skipped rather than green.
Sample output of a local run: