Skip to content

Latest commit

 

History

History
230 lines (162 loc) · 9.98 KB

Testing.md

File metadata and controls

230 lines (162 loc) · 9.98 KB

Testing

Drill makes extensive use of JUnit and other libraries for testing. This page provides pointers to the information you need to work with Drill tests. We don't repeat that information; you will want to follow the links and read the original material to get a complete understanding of the libraries that Drill uses.

Writing Tests

General Testing Tools

Deprecated Drill Testing Techniques

This is a list of old Drill testing machinery that we have cleaner machinery for now.

Legacy Drill Testing Techniques

This is a list of old Drill testing machinery for which there is no other alternative at the moment.

Latest Drill Testing Techniques

These are all the latest Drill testing Techniques that you should use. If you find one of these approaches insufficient for your use, please enhance them or file a Jira so that they an be improved.

Testing Guidelines

When testing an operator or a specific internal mechanism in Drill you should use unit tests and should avoid depending on integration tests. This approach means that you will have to design your implementation to be decoupled from the rest of Drill. While this seems daunting, in reality it doesn't take much time and you will spend much less time debugging and stressing about customer bug fixes.

Drill provides several tools to facilitate unit testing. See:

The following are some examples of operators which have leveraged these techniques for unit testing:

After the unit tests are implemented and passing, use integration tests to ensure the whole system works. The standard tools for accomplishing integration testing are:

The general rule is: test as close to your code as possible, refactoring and breaking dependencies where needed to accomplish this. On the other hand, if the test is more of a SQL or planner-level concept, then testing at the query level might be fine.

Categories

To mark a test with a category you can do the following:

@Category(SlowTest.class)
public class MyTest {
  // Testing code
}

To mark a test with multiple categories you can do the following:

@Category({SlowTest.class, SecurityTest.class})
public class MyTest {
  // Testing code
}

Running Tests

Drill tests run in parallel. The model for parallel execution is to divide test classes between multiple forked test processes. Each test process then runs the test classes assigned to it sequentially.

Speeding Up Test Runs

There are a couple knobs you can turn to make tests run faster on your machine.

  • Maven Build Threads: -T <num>
  • Sure Fire Fork Count: -DforkCount=<num>
  • Test Categories

-T

Maven allows you to use multiple threads to compile sub modules. Also when running tests each build thread forks its own surefire process, so the tests for different submodules are run in parallel. In order to leverage this use the -T flag. By default this option is effectively 1, so there is only one build thread by default.

Ex. In order to run the build using two maven threads use the following command.

mvn -T 2 clean install

-DforkCount

To run tests within a submodule in parallel you can use the -DforkCount option. By default this 2, so two surefire processes are forked for each build thread.

Ex. Run 4 test processes in parallel

mvn clean install -DforkCount=4

Note: The -DforkCount option interacts with -T. When used together each build thread (-T) gets -DforkCount test processes.

Running Categories

You can leverage categories to run subsets of the tests. This is useful if you need to test something like a storage plugin on a remote jenkins server. See the java docs here for examples. For a list of all the availabe categories go here.

Maven

It is often helpful to know the set of properties that Maven defines and are available for use in the POM file.

Surefire Plugin

Drill uses the Maven Surefire plugin to run tests. The root POM contains significant test configuration that you may or may not want to include when running tests in a debugger:

          <configuration>
            <argLine>-Xms512m -Xmx3g -Ddrill.exec.http.enabled=false
              -Ddrill.exec.sys.store.provider.local.write=false
              -Dorg.apache.drill.exec.server.Drillbit.system_options=\
               "org.apache.drill.exec.compile.ClassTransformer.scalar_replacement=on"
              -Ddrill.catastrophic_to_standard_out=true
              -XX:MaxPermSize=512M -XX:MaxDirectMemorySize=3072M
              -Djava.net.preferIPv4Stack=true
              -Djava.awt.headless=true
              -XX:+CMSClassUnloadingEnabled -ea</argLine>
            <forkCount>${forkCount}</forkCount>
            <reuseForks>true</reuseForks>
            <additionalClasspathElements>
              <additionalClasspathElement>./exec/jdbc/src/test/resources/storage-plugins.json</additionalClasspathElement>
            </additionalClasspathElements>

You can run individual tests as follows. First, cd into the project that contains the test. Then, run the test. (You'll get an error if you try to run the test from the root Drill directory.)

cd /path/to/drill/project
mvn surefire:test -Dtest=TestCircle

Java Assertions

Drill code makes sparse use of Java exceptions, preferring the Guava equivalents. One area that assertions (-ea option) does affect is performance: queries run about 10x slower with assertions enabled due to allocator logging. When doing any performance-related work, ensure that assertions are not enabled.

class DrillBuf ...
  public boolean release(int decrement) {
    ...
    if (BaseAllocator.DEBUG) {
      historicalLog.recordEvent("release(%d). original value: %d", decrement, refCnt + decrement);
    }

Simulating GitHub Actions

Sometimes a test failure that occurs in the GitHub Actions CI cannot be reproduced locally. You can try to run the tests in a more similar environment using https://github.com/nektos/act

  1. Install act according to its instructions
  2. Edit .github/workflows/ci.yml to remove extra "matrix" elements and adjust the maven command line to set the tests you are interested in: run: mvn install -Dsurefire.useFile=false -DtrimStackTrace=false -Dtest=org.apache.drill.exec.server.rest.TestQueryWrapper -DfailIfNoTests=false
  3. Run act -P ubuntu-latest=nektos/act-environments-ubuntu:18.04 --job build to run the build

IntelliJ

IntelliJ IDEA has decent support for maven built in, IntelliJ IDEA users can use these steps to get up and running:

First, import maven pom.xml as projects.

Change settings in File > Settings > Build, Execution, Deployment > Build Tools > Maven:

  1. Enable "Work Offline" to avoid checking for dependency updates on every run
  2. Increase the "Thread Count" to speed up builds - a reasonable default is the number of CPU cores you have
  3. Go to the "Runner" subsection
  4. Enable "Delegate IDE build/run actions to Maven". The IDE didn't seem to do things in a manner compatible with running the application
  5. Enable "skip tests" you will probably want to run tests manually rather than every time you try to run something
  6. To skip some uninteresting build steps, you can also set the following properties to true: checkstyle.skip, findbugs.skip, license.skip, maven.javadoc.skip, rat.skip, skipIfEmpty

At this point you should be able to use the built-in IDE features to run tests

Running/debugging Drillbit is also possible:

  1. Create a new run configuration on the class org.apache.drill.exec.server.Drillbit
  2. you have to create a run configuration and set the VM parameters:
-Xms4G
-Xmx4G
-XX:MaxDirectMemorySize=8G
-XX:ReservedCodeCacheSize=1G
-Ddrill.exec.enable-epoll=false
-XX:MaxPermSize=512M
-XX:+CMSClassUnloadingEnabled
-XX:+UseG1GC
-Dlog.path=/opt/drill/log/drillbit.log
-Dlog.query.path=/opt/drill/log/drillbit_queries.json
-Xbootclasspath/a:/opt/drill/conf
  1. Set environment variables:
DRILL_HOST_NAME=localhost:8047
DRILL_LOG_DIR=/opt/drill/log
  1. Edit /opt/drill/conf/drill-override.conf to set some options:
drill.exec: {
  zk.connect: "localhost:2181",
  allow_loopback_address_binding: true,
}
  1. Install and run ZooKeeper on port 2181 (refer to ZooKeeper docs for this)
  2. Run the Drillbit. Note that which storage plugins are available depends on your classpath.