Implemented Watch mode filter by filename and by test name #1530

[mmulet]

Hi!
This pull request implements Watch mode filter by filename and by test name from issue #1530

Watch this short video for a demonstration of the feature:

Watch_Mode_Demo.mov

Overview

• Added new tests for this feature in test/watch-mode/basic-functionality.js
• Added documentation in the docs/recipes/watch-mode.md
• Feature works as requested in Watch mode filter by filename and by test name #1530. Namely:
• Press p to filter by a filename regex pattern.
• Press t to filter by a test name regex pattern.
• Press q to quit watch mode.
• Press Enter to trigger a test run.
• Reruns the filtered list of tests after applying filter (just like jest does)
• Use a to rerun all tests.

Summary of new feature

(taken from the docs)

Filter tests while watching

You may also filter tests while watching by using the cli. For example, after running

$ npx ava --watch

You will see a prompt like this :

Type `p` and press enter to filter by a filename regex pattern
	[Current filename filter is $pattern]
Type `t` and press enter to filter by a test name regex pattern
	[Current test filter is $pattern]

[Type `a` and press enter to run *all* tests]
(Type `r` and press enter to rerun tests ||
	Type \`r\` and press enter to rerun tests that match your filters)
Type `u` and press enter to update snapshots

command > 

So, to run only tests numbered like

• foo23434
• foo4343
• foo93823

You can type t and press enter, then type foo\d+ and press enter.
This will then run all tests that match that pattern.
Afterwards you can use the r command to run the matched tests again,
or a command to run all tests.

Code Overview

In the contributing guidelines, you say that you prefer shorter pull requests, but this is a bit long. So, to help speed things along, I'm going to provide an overview for each part that I added or changed.

1. The root of the feature starts in lib/watcher.js

I add a new variable called watchModeSkipTests
https://github.com/mmulet/ava/blob/78a3768fbe3360930351374f854f9cc7a46fda51/lib/watcher.js#L104

which is an instance of WatchModeSkipTests lib/watch-mode-skip-tests.js, this class handles the regex for the skipping, and most of the code will be passing this object around, populating its fields, etc.

There is one particular part of this file that I want to draw attention to.

#OnDataGenerator

Continue down the plan function until you come to the const _class;
This class handles the prompt for commands at watch mode. It does this by listening to stdin's data event in an asynchronous generator.

I switched to using an asynchronous generator rather than just plain old stdin.on('data') because some new commands (namely p, and t the ones where you enter regex), also need to listen to 'data'. By this I mean that they have to listen for data while processing the command. In other words, without the generator you would need an explicit state machine which can be hard to debug and keep track of.

To see what I mean, look at listenForCommand:

#data;
constructor() {
	this.#data = this.#onDataGenerator();
	this.#listenForCommand();
}
#onDataGenerator = async function * () {
...
}

#listenForCommand = async () => {
	for await (const data of this.#data) {
		await this.#onCommand(data);
	}
};

Note that the for await waits on #data, not on this.#onDataGenerator()! What this means is that nested inside the #onCommand function we can await more lines of input, like this:

#onCommand = async (data) => {
  switch(data) {
     case 't':
        promptTheUserForSomething()
        const dataFromUser = await this.#data.next();
        askTheUserAnotherQuestion();
        const dataFromUserAgain = await this.#data.next();
        ...

Then when you are done processing your command, ie, the this.#onCommand returns, you can run the next loop (by getting the next this.#data) and everything works as it should.

#listenForCommand = async () => {
        //This works just as you think it would,
        // each data is a new command, not
        // a prompt or something having to do with
        // inner state
	for await (const data of this.#data) {
		await this.#onCommand(data);
	}
};

And, that's how it works.
It's not too complicated, but maybe a bit unusual. So, I just wanted to document my choices and communicate clearly about why I made them.

The rest of the addition to the plan function is as you would expect, it asks the user for the command and prompts for regex filters if necessary.

Oh the places you'll go

As I mentioned before, the rest of the code is mostly passing around the watchModeSkipTests.

• From the plan function, the watchModeSkipTests, gets added to instructions which is yielded
• The yield ends up at the async for loop in start, which means it gets rolled up in the runtimeOptions variable. Which is the passed to api.run
• From inside api.run, the watchModeSkipTestsOrUndefined branches out to 2 different places

1. First is to the fork options, which passes it to the fork and in turn the worker. Note that the fork options is watchModeSkipTestsData because only the Data gets passed to the worker, not the class itself.
2. Then, it gets emitted as part of the 'run' event. This run event will be consumed by the reporter, more on that later.

Since we are branching, let's choose a path

To the worker

• So, after api.run, we can let the existing code pass all the options to the worker, it's only in [worker/base.js]
  (https://github.com/mmulet/ava/blob/78a3768fbe3360930351374f854f9cc7a46fda51/lib/worker/base.js#L75) that we need to pass the Data to the Runner
• Once we are in the runner, we've reached the skipTests 1 of 2 destinations
  It is inside the runner that we use the watchModeSkipTests to actually , well, skip tests. The logic is simple, on every chain check if we should skip the test, and if we should, we skip!

Another Side, Another Story

Meanwhile, we have a second task, not just skip a test from running, we also want to prevent its result from being shown at all (this is a filter, the whole idea is we have too many tests to sift through).

• Let's pick up from where we left off, being emitted by emit('run'), this brings us to the cli.js, all the arguments for run are in the plan variable which gets sent to the reporter with the reporter.startRun(plan).
• At startPlan, we've reached our destination, the reporter.
• Now, recall from our other destination (the runner) that we skip all tests that have been filtered out (don't match our regexes). So to prevent the output from being displayed, we look at all tests being marked [skip] and filter out those that are being skipped due to our filters

And there you have it, a complete tour of the changes!

About the tests

One thing you'll notice while reviewing the tests is that I added try catch blocks on each and every one . I did this because whenever a test would fail, the entire test would just hang and end in a timeout (presumably because it throws an error, and the this.done function never gets run).

I tried this with a simple example, and it also had the same result:

test('basic_test', withFixture('basic'), async (t, fixture) => {
	await fixture.watch({
		async 1({process}) {
			t.is(1,2); //should fail
			this.done();
		}
	});

Your contributing guidelines say not to include unrelated code changes in the pull request, so I'm considering this error as out of scope for now. But, now you know why the try catch blocks are there (the CLI will still report a t.$someFunc() failure as a failure, it just won't report exceptions as errors, throw $someError will be caught)

Last but not least

I'm going to submit this pull request for the bounty on https://oss.issuehunt.io/r/avajs/ava/issues/1530, but any extra tips (you
can use GitHub Sponsor page) would be very welcome!

---

IssueHunt Summary

Referenced issues

This pull request has been submitted to:

• #1530: Watch mode filter by filename and by test name

---

[novemberborn]

Hey @mmulet, this sounds really great! I'm a bit short on time but can hopefully have a look in the next few days.