Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,10 +144,11 @@ must be saved before running.
paths on backslashes only and assume a minimum path depth. Behaviour on macOS
and Linux is untested and likely degraded.

**Two engine analyzers are impaired.** `Dependent Test` cannot report under any
input, and `Resource Optimism` only detects the inline `new File(...)` form, not
the assigned-variable case. Both are defects in engine code, which project
constraints prohibit modifying. Root causes are documented in
**One engine analyzer is impaired.** `Resource Optimism` only detects the inline
`new File(...)` form, not the assigned-variable case. This is a defect in engine
code, which project constraints prohibit modifying. `Dependent Test` had the same
status until it was corrected under an approved exception — see
`documentation/srs-exceptions.md`. Root causes are documented in
`documentation/regression-baseline.md`.

### Developer setup
Expand Down
8 changes: 5 additions & 3 deletions documentation/architecture-analysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -269,9 +269,11 @@ assigned-variable case from firing; a `String`-to-`Stream` comparison in
`DependentTest` that makes the detector unreachable under any input; and a stray
`System.out.println` in `ResourceOptimism`.

**Out of scope.** Recorded in `documentation/regression-baseline.md` rather than
fixed. `DependentTest` being unreachable is why corpus coverage is capped at 20 of
21 detectors.
**Out of scope**, with one exception. Three of the four are recorded in
`documentation/regression-baseline.md` rather than fixed. The `DependentTest`
defect was corrected under an exception to the engine constraint granted by the
client on 16 November 2026, raising corpus coverage from 20 to 21 of 21
detectors; see `documentation/srs-exceptions.md`.

---

Expand Down
34 changes: 23 additions & 11 deletions documentation/regression-baseline.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,16 +67,18 @@ reflect engine versions rather than plugin defects.
| 18 | Resource Optimism | detected | detected | yes |
| 19 | Magic Number Test | detected | detected | yes |
| 20 | Verbose Test | detected | detected | yes |
| 21 | Dependent Test | not reported | not reported | yes (see below) |
| 21 | Dependent Test | detected | detected | yes |

**Coverage: 20 of 21 detectors exercised. Plugin and baseline agree on all 21.**
**Coverage: 21 of 21 detectors exercised. Plugin and baseline agree on all 21.**

Smells 1–19 are triggered by `testCalculator.java`; Verbose Test is isolated in
`testVerbose.java` because its threshold requires a deliberately oversized
method that would otherwise dominate the readable fixture.

Dependent Test is absent from both. This is not a corpus gap — the analyzer
cannot fire under any input (see Finding 3).
Dependent Test is triggered by `testAccount.java`, where
`testBalanceAfterDeposit()` calls `testDeposit()`. The detector could not fire
at all until the defect in Finding 3 was corrected under an approved SRS
exception; see `documentation/srs-exceptions.md`.

## Discrepancy found and resolved: Constructor Initialization

Expand Down Expand Up @@ -141,19 +143,29 @@ without an `exists()` check — is never flagged. Only the inline form, which
correctly uses `.asString().equals("File")`, fires. The corpus triggers the
inline form deliberately.

### Finding 3 — Dependent Test cannot fire
### Finding 3 — Dependent Test could not fire — **corrected**

`DependentTest` compares a `String` to a `Stream`:
`DependentTest` compared a `String` to a `Stream`:

```java
x.getName().equals(testMethods.stream().map(z -> z.getMethodDeclaration().getNameAsString()))
```

This is always false. A corrected version exists in the same file, commented
out. Independently, the `TestMethod` element the analyzer would create never has
`setHasSmell(true)` called on it, so even a matching condition would report
nothing. The detector is inoperative under all inputs, which is why 20 of 21 is
the maximum attainable corpus coverage.
This is always false, so the analyzer's element list was never populated. A
corrected version existed in the same file, commented out. Independently, the
`TestMethod` element the analyzer would create never had `setHasSmell(true)`
called on it, so even a matching condition would have reported nothing. Either
defect alone made the detector inoperative under all inputs.

Both were corrected under an approved exception to the SRS engine constraint,
granted by the client on 16 November 2026. The membership test now uses
`contains()` against the set of test method names, `setHasSmell` is called, and
a method calling itself is excluded as recursion rather than dependency.

Verified on `fixtures/testAccount.java`: the unmodified engine reported
`Dependent Test = false`, the corrected engine reports `true`, on identical
input. The shared cross-IDE corpus is unaffected — no case in it contains a test
that calls another test. Full record in `documentation/srs-exceptions.md`.

### Finding 4 — Debug output on stdout

Expand Down
105 changes: 105 additions & 0 deletions documentation/srs-exceptions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Approved Exceptions to the SRS Engine Constraint

The SRS places one technical constraint on this project:

> The existing tsDetect engine must remain unchanged; the plugins must integrate
> with the current implementation rather than replacing its detection algorithms.

Every file under `src/main/java/testsmell/` outside the `plugin/` package is
therefore off limits, and `.github/workflows/eclipse-ci.yml` enforces this
mechanically: it verifies all engine source files against
`tools/corpus/engine-checksums.txt` before compiling, and fails the build on any
difference.

This document records the exceptions the client has approved, so that a checksum
change can always be traced to a decision rather than an accident.

---

## Exception 1 — DependentTest detector

| | |
|---|---|
| **File** | `src/main/java/testsmell/smell/DependentTest.java` |
| **Requested** | 9 November 2026 |
| **Approved** | 16 November 2026 |
| **Approved by** | Mohamed Wiem Mkaouer (client) |
| **Scope** | This file only |

### Why it was requested

The Dependent Test detector could not report a smell under any input. Two
independent defects, either of which alone was sufficient:

**The membership test compared a `String` to a `Stream`.**

```java
x.getName().equals(testMethods.stream().map(z -> z.getMethodDeclaration().getNameAsString()))
```

A `String` is never equal to a `Stream`, so the condition was false for every
method in every file and the analyzer's element list was never populated.

**The smell flag was never set.** Even on a match, the element was added without
`setHasSmell(true)`, and `getHasSmell()` counts only elements whose flag is set.

A corrected implementation was present in the same file, commented out beneath
the broken code.

### Client decision

> Please proceed with the eight-line fix as an approved SRS exception. Since the
> Dependent Test detector cannot fire under any input, documenting it only as a
> limitation would leave a required feature unusable.

Reporting the defect upstream to the tsDetect maintainers was approved as a
secondary action, not a substitute for correcting the delivered version.

### What changed

`DependentTest.runAnalysis` only. Three corrections:

1. Membership is tested with `contains()` against a `Set` of the test method
names declared in the file, rather than comparing a name to a `Stream`.
2. `setHasSmell` is called with the result for every test method examined,
matching the convention the other analyzers follow.
3. A method calling itself is excluded — that is recursion, not a dependency
between tests.

Two imports were added (`java.util.HashSet`, `java.util.Set`). The commented-out
original implementation was removed. No other engine file was touched, and no
detection logic outside this analyzer was altered.

### Validation

A regression fixture was added at `fixtures/testAccount.java` /
`fixtures/Account.java`, in which `testBalanceAfterDeposit()` calls
`testDeposit()`.

| Engine | Result on identical input |
|---|---|
| Unmodified | `Dependent Test = false` |
| Corrected | `Dependent Test = true` |

Full regression corpus re-run after the change:

- Fixture corpus coverage rises from **20 of 21** to **21 of 21** detectors.
- All other detector results are unchanged.
- The shared cross-IDE corpus is unaffected: 5 matched, 5 declared divergences,
0 unexpected, 0 stale. No case in that corpus contains a test calling another
test, so no expected result moves and no new divergence appears.

`tools/corpus/engine-checksums.txt` was regenerated to cover the corrected file.
This is the only reason the engine checksums differ from the inherited engine.

### Not changed

The three remaining engine defects recorded in
`documentation/regression-baseline.md` are untouched and remain out of scope:

- Windows-only path splitting in `TestFile.getTestFileName()` and `getTagName()`
- `ResourceOptimism` comparing a `Type` to a `String`, so its assigned-variable
case never fires
- A stray `System.out.println` in `ResourceOptimism`

No exception has been requested or granted for any of these.
7 changes: 3 additions & 4 deletions documentation/use-cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -312,10 +312,9 @@ is skipped and named in a dialog; the rest of the project is still analyzed.
Pairing happens in the plugin rather than the engine, so the recognised set can be
extended without touching engine code.

**Dependent Test is never reported.** The detector compares a `String` to a
`Stream`, which is always false, and never sets its smell flag. It cannot fire
under any input. This is an engine defect, documented in
`documentation/regression-baseline.md` rather than fixed.
**Resource Optimism is partially impaired.** It detects only the inline
`new File(...)` form, not the assigned-variable case. This is an engine defect,
documented in `documentation/regression-baseline.md` rather than fixed.

**Detection is project-wide.** UC2 and UC5 analyze an entire project. There is no
use case for analyzing a single file on demand; UC1 will cover the per-file case
Expand Down
12 changes: 12 additions & 0 deletions fixtures/Account.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
public class Account {

private int balance;

public void deposit(int amount) {
balance += amount;
}

public int getBalance() {
return balance;
}
}
24 changes: 24 additions & 0 deletions fixtures/testAccount.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import static org.junit.Assert.*;

import org.junit.Test;

/**
* Fixture for the Dependent Test detector: testBalanceAfterDeposit calls
* testDeposit, so the two only pass in a particular order.
*/
public class testAccount {

@Test
public void testDeposit() {
Account account = new Account();
account.deposit(50);
assertEquals(50, account.getBalance());
}

@Test
public void testBalanceAfterDeposit() {
testDeposit();
Account account = new Account();
assertEquals(0, account.getBalance());
}
}
30 changes: 19 additions & 11 deletions src/main/java/testsmell/smell/DependentTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@

import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

public class DependentTest extends AbstractSmell {

Expand Down Expand Up @@ -49,20 +51,26 @@ public void runAnalysis(CompilationUnit testFileCompilationUnit, CompilationUnit
classVisitor = new DependentTest.ClassVisitor();
classVisitor.visit(testFileCompilationUnit, null);

// Every test method name declared in this file.
Set<String> testMethodNames = new HashSet<>();
for (TestMethod testMethod : testMethods) {
if (testMethod.getCalledMethods().stream().anyMatch(x -> x.getName().equals(testMethods.stream().map(z -> z.getMethodDeclaration().getNameAsString())))){
smellyElementList.add(new main.java.testsmell.TestMethod(testMethod.getMethodDeclaration().getNameAsString()));
}
testMethodNames.add(testMethod.getMethodDeclaration().getNameAsString());
}

/*
for (int i = 0; i < testMethods.get(i).getCalledMethods().size(); i++) {
for (TestMethod testMethod : testMethods) {
if (testMethods.get(i).getCalledMethods().stream().anyMatch(x -> x.getName().equals(testMethod.getMethodDeclaration().getNameAsString()))) {
smellyElementList.add(new testsmell.TestMethod(testMethod.getMethodDeclaration().getNameAsString()));
}
}
}*/
// A test is dependent when it calls one of the OTHER test methods in the
// same class. A call to itself is recursion, not a dependency.
for (TestMethod testMethod : testMethods) {
String ownName = testMethod.getMethodDeclaration().getNameAsString();

boolean callsAnotherTest = testMethod.getCalledMethods().stream()
.anyMatch(called -> !called.getName().equals(ownName)
&& testMethodNames.contains(called.getName()));

main.java.testsmell.TestMethod element =
new main.java.testsmell.TestMethod(ownName);
element.setHasSmell(callsAnotherTest);
smellyElementList.add(element);
}
}

/**
Expand Down
2 changes: 1 addition & 1 deletion tools/corpus/engine-checksums.txt
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ d61b074e665c0e104bc867aae0b9ef8eefb0e6cb26b27a0e2c85bd472e2745c4 src/main/java/t
bd291cf0b3f0c3aa568f7edf767d3ab60ff55f872fa264ab610e9294d68d4703 src/main/java/testsmell/smell/ConditionalTestLogic.java
7d04ea1125812dd8ab66f844d2069ea4b950532949bcb2208eeb68933bb36268 src/main/java/testsmell/smell/ConstructorInitialization.java
67da93a6cd226962cff71f80af9e57a0c46146a76a17bce445d2e157ddd4967a src/main/java/testsmell/smell/DefaultTest.java
8725ebcf4604ce9a0fb81a8ef23f4c5ccd84a01b336100b25b80924c9b6c1ced src/main/java/testsmell/smell/DependentTest.java
a37bbe11d23802e508546bc5a53bc9cd565db25815a66e669d9cc72b006ace09 src/main/java/testsmell/smell/DependentTest.java
84efa4fe0d38ccd3329151f37d99c0816e8c983547338995389c6b986f8d3453 src/main/java/testsmell/smell/DuplicateAssert.java
488789ac5b5fff83474a4f61363e08f0059bb903d815369e51b77a80b6b1af1b src/main/java/testsmell/smell/EagerTest.java
68aca3ea71f4fe19aca82ad74ada44bb57f03b9e1f0eec470d671115efea4561 src/main/java/testsmell/smell/EmptyTest.java
Expand Down
Loading