How to correctly implement and test Custom Lint Rules in Android Studio?

Question

I'm following this tutorial and this Custom Detector Example in order to implement Custom Lint Rules. Basically what I've done is:

1. Create a new Android Project in Android Studio;
2. Create a java module for project created in step 1;
3. On module's build.gradle, import Lint API dependencies;
4. Create an Issue & IssueRegistry & CustomDetector;
5. Reference the IssueRegistry on module's build.gradle;
6. Create Unit tests;

My problem is, during the execution of my JUnits, I always receive "No Warning". When I debug the test, I can see that my Custom Detector isn't called, what am I doing wrong?

Strings.java

public class Strings {

    public static final String STR_ISSUE_001_ID = "VarsMustHaveMoreThanOneCharacter";
    public static final String STR_ISSUE_001_DESCRIPTION = "Avoid naming variables with only one character";
    public static final String STR_ISSUE_001_EXPLANATION = "Variables named with only one character do not pass any meaning to the reader. " +
        "Variables name should clear indicate the meaning of the value it is holding";
}

Issues.java

public class Issues {

    public static final
    Issue ISSUE_001 = Issue.create(
            STR_ISSUE_001_ID,
            STR_ISSUE_001_DESCRIPTION,
            STR_ISSUE_001_EXPLANATION,
            SECURITY,
            // Priority ranging from 0 to 10 in severeness
            6,
            WARNING,
            new Implementation(VariableNameDetector.class, ALL_RESOURCES_SCOPE)
    );
}

IssuesRegistry.java

public class IssueRegistry extends com.android.tools.lint.client.api.IssueRegistry {
    @Override
    public List<Issue> getIssues() {
        List<Issue> issues = new ArrayList<>();
        issues.add(ISSUE_001);
        return issues;
    }
}

VariableNameDetector.java

public class VariableNameDetector extends Detector implements Detector.JavaScanner {

    public VariableNameDetector() {

    }

    @Override
    public boolean appliesToResourceRefs() {
        return false;
    }

    @Override
    public boolean appliesTo(Context context, File file) {
        return true;
    }

    @Override
    @Nullable
    public AstVisitor createJavaVisitor(JavaContext context) {
        return new NamingConventionVisitor(context);
    }

    @Override
    public List<String> getApplicableMethodNames() {
        return null;
    }

    @Override
    public List<Class<? extends Node>> getApplicableNodeTypes() {
        List<Class<? extends Node>> types = new ArrayList<>(1);
        types.add(lombok.ast.VariableDeclaration.class);
        return types;
    }

    @Override
    public void visitMethod(
            JavaContext context,
            AstVisitor visitor,
            MethodInvocation methodInvocation
    ) {
    }

    @Override
    public void visitResourceReference(
            JavaContext context,
            AstVisitor visitor,
            Node node,
            String type,
            String name,
            boolean isFramework
    ) {
    }

    private class NamingConventionVisitor extends ForwardingAstVisitor {

        private final JavaContext context;

        NamingConventionVisitor(JavaContext context) {
            this.context = context;
        }

        @Override
        public boolean visitVariableDeclaration(VariableDeclaration node) {
            StrictListAccessor<VariableDefinitionEntry, VariableDeclaration> varDefinitions =
                    node.getVariableDefinitionEntries();

            for (VariableDefinitionEntry varDefinition : varDefinitions) {
                String name = varDefinition.astName().astValue();
                if (name.length() == 1) {
                    context.report(
                            ISSUE_001,
                            context.getLocation(node),
                            STR_ISSUE_001_DESCRIPTION
                    );
                    return true;
                }
            }
            return false;
        }
    }
}

build.gradle

apply plugin: 'java'

configurations {
    lintChecks
}

ext {
    VERSION_LINT_API = '24.3.1'
    VERSION_LINT_API_TESTS = '24.3.1'
}

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation "com.android.tools.lint:lint-api:$VERSION_LINT_API"
    implementation "com.android.tools.lint:lint-checks:$VERSION_LINT_API"
    testImplementation "com.android.tools.lint:lint-tests:$VERSION_LINT_API_TESTS"
}

jar {
    manifest {
        attributes('Lint-Registry': 'br.com.edsilfer.lint_rules.resources.IssueRegistry')
    }
}

sourceCompatibility = "1.7"
targetCompatibility = "1.7"

TestVariableNameDetector.java

private static final String ARG_DEFAULT_LINT_SUCCESS_LOG = "No warnings.";

    @Override
    protected Detector getDetector() {
        return new VariableNameDetector();
    }

    @Override
    protected List<Issue> getIssues() {
        return Collections.singletonList(Issues.ISSUE_001);
    }

    public void test_file_with_no_variables_with_length_equals_01() throws Exception {
        assertEquals(
                ARG_DEFAULT_LINT_SUCCESS_LOG,
                lintProject(java("assets/Test.java", "public class Test {public String sampleVariable;}"))
        );
    }

    public void test_file_with_variables_with_length_equals_01() throws Exception {
        assertEquals(
                ARG_DEFAULT_LINT_SUCCESS_LOG,
                lintProject(java("assets/Test3.java", "public class Test {public String a;bnvhgvhj}"))
        );
    }
}

P.S.: on Java's module I do not have access to assetsor res folder, that is the reason why I've created a String.java and I'm using java(to, source) in my Unit test - I assumed that this java method does the same as the xml from the tutorial link I referenced at the top of this question.

Answer 1

It turned out that in my case the problem was with the JUnit itself. I think that the way I was attempting to simulate the file was wrong. The text below is part of the README.md of a sample project that I've created in order to document what I've learned from this API and answers the question in the title:

---

Create

1. Create a new Android Project;
2. Create a new Java Library Module - Custom Lint Rules are packaged into .jar libraries once they are ready, therefore the easiest way to implement them using them is inside a Java Module Library;
3. On module's build.gradle:
   • add target and source compatibility to Java 1.7;
   • add dependencies for lint-api, lint-checks and test dependencies;
   • add jar packing task containing two attributes: Manifest-Version and Lint-Registry, set the first to 1.0 and the second as the full path to a class that will later on contain the issue's catalog;
   • add a default tasl assemble;
   • [OPTIONAL]: add a task that will copy the generated .jar into ~/.android/lint;
4. Check REF001 and choose a Detector that best suits your needs, based on it create and implement a class to fulfill the Detector's role;
5. Still based on REF0001 chosen file, create and implement a Checker class, later referring to it inside Detector's createJavaVisitor() method;
   • for the sake of SRP, do not place Checker in the same file of Detector's class;
6. Copy the generated .jar file from build/lib to ~/.android/lint - if you added a task on build.gradle that does this you can skip this step;
7. Restart the computer - once created and moved into ~/.android/lint, the Custom Rules should be read by Lint next time the program starts. In order to have the alert boxes inside Android Studio, it is enough to invalidate caches and restart the IDE, however, to have your custom rules caught on Lint Report when ./gradlew check, it might be necessary to restart your computer;

Testing Detectors and Checkers

Testing Custom Rules is not an easy task to do - mainly due the lack of documentation for official APIs. This section will present two approaches for dealing with this. The main goal of this project is to create custom rules that will be run against real files, therefore, test files will be necessary for testing them. They can be places in src/test/resources folder from your Lint Java Library Module;

Approach 01: LintDetectorTest

1. Make sure you've added all test dependencies - checkout sample project's build.gradle;
2. Copy EnhancedLintDetectorTest.java and FileUtils.java into your project's test directory;
   • There is a known bug with Android Studio that prevents it from seeing files from src/test/resources folder, these files are a workaround for that;
   • EnhancedLintDetectorTest.java should return all issues that will be subject of tests. A nice way to do so is getting them from Issue Registry;
3. Create a test class that extends from EnhancedLintDetectorTest.java;
4. Implement getDetector() method returning an instance of the Detector to be tested;
5. Use lintFiles("test file path taking resources dir as root") to perform the check of the Custom Rules and use its result object to assert the tests;

Note that LintDetectorTest.java derives from TestCase.java, therefore, you're limited to JUnit 3.

Approach 02: Lint JUnit Rule

You might have noticed that Approach 01 might be a little overcomplicated, despite the fact that you're limited to JUnit 3 features. Because of that GitHub user a11n created a Lint JUnit Rule that allows the test of Custom Lint Rules in a easier way that counts with JUnit 4 and up features. Please, refer to his project README.md for details about how to create tests using this apprach.

Currently, Lint JUnit Rule do not correct the root dir for test files and you might no be able to see the tests passing from the IDE - however it works when test are run from command line. An issue and PR were created in order to fix this bug.