Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 23 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,23 +105,34 @@ engine code is out of scope for this plugin.

### File naming convention

The plugin pairs a test file with its production file by removing the first four
characters of the test filename. A file named `testExample.java` is matched to
`Example.java`.
The plugin pairs a test file with the production file it exercises by name. These
conventions are recognised:

The widely used `ExampleTest.java` convention is **not** supported — it produces
no match, and the run reports a file-pairing error. This is inherited behaviour
from the detection engine. See Known limitations.
| Test file | Production file |
|---|---|
| `testExample.java` | `Example.java` |
| `TestExample.java` | `Example.java` |
| `ExampleTest.java` | `Example.java` |
| `ExampleTests.java` | `Example.java` |
| `ExampleTestCase.java` | `Example.java` |
| `ExampleIT.java` | `Example.java` |

Matching is by file name, so the production file may live anywhere in the project
— a Maven layout with `src/main/java` and `src/test/java` works.

A test file matching none of these is skipped and named in a dialog at the end of
the run. The rest of the project is still analyzed.

### Known limitations

**Test file naming.** Only the `testExample.java` → `Example.java` convention is
supported (see above). Projects using `ExampleTest.java` cannot be analyzed.
**Test file naming.** Test files must follow one of the recognised conventions
listed above. A file named in some other way cannot be paired with its production
file and is skipped, with a dialog naming it.

**Analysis runs on the UI thread.** The workbench does not repaint while
detection runs. This is imperceptible at typical project sizes — roughly 750 ms
for 200 test files — but would be noticeable on a very large codebase. See
`documentation/performance.md`.
**Test methods are found by name.** A method counts as a test when its name starts
with `test` or `Test`. JUnit 5 tests named for behaviour rather than prefixed —
`shouldReturnZeroWhenEmpty()`, say — are not recognised, even when annotated
`@Test`.

**Detection is on demand.** The plugin does not analyze automatically when a
file is saved. Analysis must be triggered explicitly.
Expand Down
18 changes: 11 additions & 7 deletions documentation/architecture-analysis.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,13 +245,17 @@ The engine exposes the responsible elements through
All 21 detectors always run and all results are always reported (§2.4). A team that
accepts some smells has no way to quieten them. **In scope.** → later filed as #15

### F10 — File pairing is convention-bound and undocumented
The handler pairs a test file with its production file by stripping the first four
characters of the filename, supporting only `testExample.java` → `Example.java`.
The widely used `ExampleTest.java` convention produces no match and the run fails.
**In scope** as a limitation to document; changing the convention would alter
engine expectations. → documented in `README.md` and
`documentation/regression-baseline.md`
### F10 — File pairing is convention-bound, and failure is fatal
`TSmellHandler` pairs a test file with its production file by stripping the first
four characters of the filename, supporting only `testExample.java` →
`Example.java`. `ExampleTest.java` yields a name matching nothing, and the
resulting null throws a `NullPointerException` that aborts the whole run — so one
oddly named file means a project of any size is analyzed not at all.

Pairing is decided entirely in the plugin: `TestFile` stores whichever two paths
it is given, and the engine never derives one name from the other. **In scope**,
and no engine change is involved. → fixed; the recognised conventions are listed
in `README.md`

### F11 — No headless build exists
`pom.xml` is the CLI's POM and does not build the bundle (§3.2). **In scope** but
Expand Down
22 changes: 9 additions & 13 deletions documentation/performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,19 +95,15 @@ triggering detection automatically on file save has been removed.

## Limitations

**The large corpora are synthetic.** The plugin pairs a test file with its
production file by removing the first four characters of the filename, so it
supports only the `testExample.java` → `Example.java` convention. The widely
used `ExampleTest.java` convention produces no match, and the run fails with a
file-pairing error rather than returning results.

This made benchmarking against a real open-source Java project impractical, and
is a limitation of the tool rather than of the measurement. It is recorded in
`documentation/regression-baseline.md` alongside the other engine findings.

The same constraint bounds how much the residual marker-flush cost matters in
practice: a project large enough for it to become visible is a project the
plugin cannot pair files for in the first place.
**The large corpora are synthetic.** They are generated rather than taken from a
real project, so every file is structurally similar and the smell mix is uniform.
A real codebase of the same size would vary more.

These figures were taken while the plugin supported only the
`testExample.java` → `Example.java` convention, which is why the corpora use that
naming. Pairing now recognises `ExampleTest.java` and several other conventions,
so benchmarking against a real open-source project has become practical; the
figures here have not yet been re-taken on one.

**Timing covers detection and marker creation.** Populating the results table
happens after the measured window and is not included in the figures above.
Expand Down
15 changes: 8 additions & 7 deletions documentation/use-cases.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@
## Assumptions and constraints

- The project under analysis has the Java nature enabled.
- Test files follow the `testExample.java` ↔ `Example.java` naming convention. The
`ExampleTest.java` convention is not supported — see §Limitations.
- Test files follow one of the recognised naming conventions — `testExample.java`,
`ExampleTest.java`, `ExampleTests.java`, `ExampleTestCase.java` or
`ExampleIT.java`, each paired with `Example.java`.
- The tsDetect engine is used unmodified, per the SRS technical constraint. The
plugin supplies input and presents results; it does not alter detection logic.

Expand Down Expand Up @@ -305,11 +306,11 @@ page are available.
These constrain the use cases above and are recorded here so they are not
rediscovered as defects.

**File pairing.** A test file is matched to its production file by stripping the
first four characters of the filename, so only `testExample.java` → `Example.java`
is supported. `ExampleTest.java` produces no match and the run reports a pairing
failure. Changing this would alter what the engine expects, which the SRS
constraint places out of scope.
**File pairing is by name.** A test file is matched to its production file by
name, across the conventions listed in `README.md`. A file matching none of them
is skipped and named in a dialog; the rest of the project is still analyzed.
Pairing happens in the plugin rather than the engine, so the recognised set can be
extended without touching engine code.

**Dependent Test is never reported.** The detector compares a `String` to a
`Stream`, which is always false, and never sets its smell flag. It cannot fire
Expand Down
206 changes: 147 additions & 59 deletions src/main/java/testsmell/plugin/handlers/TSmellHandler.java
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,10 @@
*/
public class TSmellHandler extends AbstractHandler {
private ArrayList<IResource> testPaths = new ArrayList<IResource>();
/** Test file resource -> the production file it was paired with. */
private HashMap<IResource, IResource> paths = new HashMap<>();
/** Test files that matched no production file, reported once after a run. */
private List<IResource> unmatchedTestFiles = new ArrayList<IResource>();
private IProject project;
private List<TestFile> testFiles = new ArrayList<TestFile>();

Expand All @@ -81,6 +84,7 @@ public Object execute(ExecutionEvent event) throws ExecutionException {
long startTime = System.currentTimeMillis();
testPaths.clear();
paths.clear();
unmatchedTestFiles.clear();
testFiles.clear();
Shell activeShell = PlatformUI.getWorkbench().getActiveWorkbenchWindow().getShell();

Expand Down Expand Up @@ -276,51 +280,79 @@ private IProject projectFromActiveEditor() {
* @throws JavaModelException
*/
private void createTestFiles(IProject project) throws CoreException, JavaModelException {
if (project.isNatureEnabled("org.eclipse.jdt.core.javanature")) {
IJavaProject javaProject = JavaCore.create(project);
IPackageFragment[] packages = javaProject.getPackageFragments();
for (IPackageFragment pkg : packages) {
if (pkg.getKind() == IPackageFragmentRoot.K_SOURCE) {
for (ICompilationUnit unit : pkg.getCompilationUnits()) {
testPaths = getTestFilePaths(unit);
}
if (!project.isNatureEnabled("org.eclipse.jdt.core.javanature")) {
return;
}

IJavaProject javaProject = JavaCore.create(project);
IPackageFragment[] packages = javaProject.getPackageFragments();
for (IPackageFragment pkg : packages) {
if (pkg.getKind() == IPackageFragmentRoot.K_SOURCE) {
for (ICompilationUnit unit : pkg.getCompilationUnits()) {
collectTestFilePath(unit);
}
}
if (testPaths.size() > 0) {
try {
getProdFilePaths();
}

if (testPaths.isEmpty()) {
return;
}

TestFile testFile;
getProdFilePaths();

for (Entry<IResource, IResource> fullPaths : paths.entrySet()) {
String prodFilePath = fullPaths.getKey().getRawLocation().toOSString();
String testFilePath = fullPaths.getValue().getRawLocation().toOSString();
testFile = new TestFile(project.getName(), testFilePath, prodFilePath);
testFiles.add(testFile);
}
} catch (NullPointerException e) {
Shell activeShell = PlatformUI.getWorkbench().getActiveWorkbenchWindow().getShell();
String errorStr = "Recheck production and test file names. "
+ "\nPlease restart your workspace after the file names have been corrected. "
+ "\nBelow are the test file/production file mappings that triggered the exception: \n\n";
for (Entry<IResource, IResource> fullPaths : paths.entrySet()) {
if (fullPaths.getKey() == null) {
errorStr += "Production File: null\n";
} else {
errorStr += "Production File: " + fullPaths.getKey().getRawLocation().toString() + "\n";
}

if (fullPaths.getValue() == null) {
errorStr += "Test File: null\n";
} else {
errorStr += "Test File: " + fullPaths.getValue().getRawLocation().toString() + " \n\n";
}
}
TSmellDialog dialog = new TSmellDialog(activeShell, "Failed to load files", errorStr);
dialog.open();
}
for (Entry<IResource, IResource> pair : paths.entrySet()) {
IResource test = pair.getKey();
IResource prod = pair.getValue();
if (test == null || prod == null) {
continue;
}
String testFilePath = test.getRawLocation().toOSString();
String prodFilePath = prod.getRawLocation().toOSString();
testFiles.add(new TestFile(project.getName(), testFilePath, prodFilePath));
}

// Unmatched files are skipped rather than aborting the run, so one oddly
// named file no longer prevents the rest of the project being analyzed.
if (!unmatchedTestFiles.isEmpty()) {
reportUnmatchedTestFiles();
}
}

/**
* Tells the user which test files could not be paired, and which naming
* conventions are recognised. Analysis of the files that did pair has already
* happened by this point.
*/
private void reportUnmatchedTestFiles() {
StringBuilder message = new StringBuilder();
message.append(unmatchedTestFiles.size());
message.append(unmatchedTestFiles.size() == 1 ? " test file could not be matched"
: " test files could not be matched");
message.append(" to a production file and was skipped.\n");
message.append("The rest of the project was analyzed normally.\n\n");
message.append("A test file is paired with the production file it exercises by name.\n");
message.append("These conventions are recognised:\n\n");
message.append(" testExample.java -> Example.java\n");
message.append(" TestExample.java -> Example.java\n");
message.append(" ExampleTest.java -> Example.java\n");
message.append(" ExampleTests.java -> Example.java\n");
message.append(" ExampleTestCase.java -> Example.java\n");
message.append(" ExampleIT.java -> Example.java\n\n");
message.append("Skipped:\n\n");

int shown = 0;
for (IResource unmatched : unmatchedTestFiles) {
if (shown == 20) {
message.append(" ...and ").append(unmatchedTestFiles.size() - shown).append(" more\n");
break;
}
message.append(" ").append(unmatched.getFullPath().toString()).append("\n");
shown++;
}

Shell activeShell = PlatformUI.getWorkbench().getActiveWorkbenchWindow().getShell();
TSmellDialog dialog = new TSmellDialog(activeShell, "Some test files were skipped", message.toString());
dialog.open();
}

/**
Expand All @@ -331,15 +363,7 @@ private void createTestFiles(IProject project) throws CoreException, JavaModelEx
* @throws JavaModelException
*/
private void getProdFilePaths() throws JavaModelException {
HashMap<String, IResource> possibleProdFileNames = new HashMap<String, IResource>();
HashMap<String, IResource> potentialProdFiles = new HashMap<>();
for (IResource test : testPaths) {

String fileName = test.getName();
int prefixLength = "test".length();
String prodFileName = fileName.substring(prefixLength, fileName.length());
possibleProdFileNames.put(prodFileName, test);
}
HashMap<String, IResource> potentialProdFiles = new HashMap<String, IResource>();

IContainer container = containerForPath(project.getFullPath());
try {
Expand All @@ -348,12 +372,72 @@ private void getProdFilePaths() throws JavaModelException {
e.printStackTrace();
}

for (Entry<String, IResource> possibleNames : possibleProdFileNames.entrySet()) {
String key = possibleNames.getKey();
IResource prodIRes = potentialProdFiles.get(key);
IResource testIRes = possibleNames.getValue();
paths.put(prodIRes, testIRes);
for (IResource test : testPaths) {
IResource prodIRes = null;

for (String candidate : productionFileCandidates(test.getName())) {
prodIRes = potentialProdFiles.get(candidate);
if (prodIRes != null) {
break;
}
}

if (prodIRes == null) {
if (!unmatchedTestFiles.contains(test)) {
unmatchedTestFiles.add(test);
}
} else {
paths.put(test, prodIRes);
}
}
}

/**
* Production file names that could correspond to a given test file name, in
* the order they should be tried.
*
* <p>
* The original implementation removed the first four characters of every test
* file name, which only works for the {@code testExample.java} convention. The
* widely used {@code ExampleTest.java} convention produced a name that matched
* nothing, and the resulting null aborted the entire run.
*
* <pre>
* testExample.java -&gt; Example.java
* TestExample.java -&gt; Example.java
* ExampleTest.java -&gt; Example.java
* ExampleTests.java -&gt; Example.java
* ExampleTestCase.java -&gt; Example.java
* ExampleIT.java -&gt; Example.java
* </pre>
*
* @param testFileName the simple file name of the test file, with extension
* @return candidate production file names, possibly empty, never null
*/
private List<String> productionFileCandidates(String testFileName) {
List<String> candidates = new ArrayList<String>();

if (testFileName == null || !testFileName.endsWith(".java")) {
return candidates;
}

String baseName = testFileName.substring(0, testFileName.length() - ".java".length());

String[] prefixes = { "test", "Test" };
for (String prefix : prefixes) {
if (baseName.length() > prefix.length() && baseName.startsWith(prefix)) {
candidates.add(baseName.substring(prefix.length()) + ".java");
}
}

String[] suffixes = { "TestCase", "Tests", "Test", "IT" };
for (String suffix : suffixes) {
if (baseName.length() > suffix.length() && baseName.endsWith(suffix)) {
candidates.add(baseName.substring(0, baseName.length() - suffix.length()) + ".java");
}
}

return candidates;
}

/**
Expand Down Expand Up @@ -408,22 +492,26 @@ private HashMap<String, IResource> findJavaFiles(IContainer container) throws Co
* @return
* @throws JavaModelException
*/
private ArrayList<IResource> getTestFilePaths(ICompilationUnit unit) throws JavaModelException {
private void collectTestFilePath(final ICompilationUnit unit) {
ASTParser parser = ASTParser.newParser(AST.getJLSLatest());
parser.setKind(ASTParser.K_COMPILATION_UNIT);
parser.setSource(unit);
final CompilationUnit cu = (CompilationUnit) parser.createAST(null);
cu.accept(new ASTVisitor() {
@Override
public boolean visit(MethodDeclaration node) {
if (node.getName().getIdentifier().startsWith("test")
|| node.getName().getIdentifier().startsWith("Test")
&& !testPaths.contains(unit.getResource())) {
// The original condition was
// startsWith("test") || startsWith("Test") && !testPaths.contains(...)
// and && binds tighter than ||, so the duplicate check only applied to
// the "Test" branch. A file with several test* methods was added once
// per method.
String methodName = node.getName().getIdentifier();
boolean looksLikeTest = methodName.startsWith("test") || methodName.startsWith("Test");
if (looksLikeTest && !testPaths.contains(unit.getResource())) {
testPaths.add(unit.getResource());
return true;
}
return false;
}
});
return testPaths;
}
}
Loading