Cucumber JVM should provide reuseable API for Options/Parsing/Compiling/Running

[laeubi]

This is related to cucumber/cucumber-eclipse#368

The JavaBackend currently has no way to access SnippetGenerator and TypeRegistry that makes it impossible for IDEs or other code to reuse the information collected by the code.

[mpkorstanje]

What kind of information are you looking for? Directly accessing the Backend might not be the best way to acquire it.

[laeubi]

In general I'd like to reuse the parsing glue as well as generation of code snippets.
Currently we parse that data by our self for the cucumber-eclipse plugin, the problem with that approach is, that this might produce inconsistencies. So it seems natural to reuse the JavaBackend for this as all the logic about finding glue-code is already there and we don't want to catch up with changes done here.

To summarize: Using the eclipse plugin should produce the same errors/output as running against cucumber-cli.

[mpkorstanje]

In general I'd like to reuse the parsing glue as well as generation of code snippets.

In general that is going to be a problem because not all glue can be parsed without executing the code. Lamda step definitions for instance require the class that defines them is instantiated. This can be a problem when building the test context is a slow process.

But lets disregard Lambda step definitions for now.

it seems natural to reuse the JavaBackend for this as all the logic about finding glue-code is already there and we don't want to catch up with changes done here.

Specifically which bits of information are you looking for? And in what format would you like to receive them? I'd rather not open up all objects and let you look inside them -- it would make Cucumber-JVM very hard to evolve -- but we could come up with a JSON format that makes it easier.

[laeubi]

I'm sure your using some kind of IDE for development (maybe Idea or Eclipse), so simply think about what would you expect from an IDE to help you write Java code. Of course it is nice to have syntax highlight, but even more helpful are features that provide you with quick-fix and suggestions for autocomplete and you want to get error/warning hints if something is probably wrong.

Especially you want to support the "Specification Guys" so the don't need to know what steps are there but can find a list on there hand, automatically generated from projects code.

Lets assume I have a feature file and want to add rich edit capabilities for the user.
So the user want's to know:

• If I run this feature with cucumber: will it produce any error (e.g. missing step defs, wrong parameter types)
• What steps are available and what parameter does it require
• Get autocompletion proposals when typing, jumping from one parameter to the next
• If a step is missing, generate me the snippet and insert it into a GlueCode file so it can be implemented

All the information and features are already available in cucumber-jvm, its just hard to access them because cucumber-jvm is specially designed for the "run once" case.
I'm not sure if it really is useful to have this as JSON since that will require another parsing step as well, I also won't expect that the API will stay stable all the time, it just must have these stuff accessible by anyone who wants to write tools for cucumber. If api change in higher versions, tools must adapt to this.

The Lamdastuff is a good example: cucumber-jvm must have some kind of logic to execute and match steps, as a tool author I don't mind HOW cucumber-jvm gather the information (even if its slow) I just don't want to implement the same logic again.

So probably I would create an instance of JavaBackend (or any other Interface!) initlize it with some options (e.g. glue packages, classloader) and then I call a method that gives me all steps that would be available the same way as if I would have run cucumber on the cli.
Next I might call a method that matches a gherkin-ast and get back any error that cucumber would report when I run it with the cli.

So to summarize: I don't wan't new exclusive features for IDE/Tools, just reuse the stuff the cucumber-cli runner will internally already do in a way that does not involve executing the feature with cli and parse the std-out of the process. To archive this I won't mind calling internal API even if that means I has to take care about changes to this internal stuff when I upgrade to a higher version.

[mpkorstanje]

I'm not interested in considering the possibility of opening up Cucumber JVM outside of well defined interfaces and API's.

However for first three use cases you've described I believe the Plugin system would be most appropriate. You can use the StepDefinedEvent to access information about the available steps. It is currently a limited set of information but I am not opposed to extending the information that is available. The Plugin system also provides the SnippetsSuggestedEvent which should cover your fourth use case.

[laeubi]

Okay so you suggest to use something like this:

Runtime runtime = cucumber.runtime.Runtime.builder().withAdditionalPlugins(new StepDefinitionReporter() {

				@Override
				public void stepDefinition(StepDefinition stepDefinition) {
					// do something with it
					
				}
			}).build();

This would be perfectly fine, i just would need more fain grained control over the "ecexute" stage, since I don't really want to execute (yet) the code, i would need something like:

• runtime.parse() (parse all step defs)
• runtime.compile(feature) (compile all steps against a CucumberFeature but don't run them)

So do you think the Runtime can be extended like this so the current run() is split into these "Phases" instead of one big "run()"?

[mpkorstanje]

If you're looking to do that then you might be better of assembling the run time components yourself like the JUnit and TestNG Runner do.

[laeubi]

That might be possible but I would prefer not to replicate all the mechnism. If JUnit an TestNG already do this, wouldn't it be valid to provide reuseable API that

• CLI
• Junit
• TestNG
• Tools/IDEs can use...

[mpkorstanje]

We used to have this design in v1 and v2. The Runtime was used directly in by the CLI, JUnit and TestNG. It was unsatisfactory.

The CLI, JUnit TestNG each impose a different structure on the way tests are discovered, organized, filtered and executed. Because each of the runners is interested in low level concepts - discovery, features, scenarios, filtering- this structure does not generalize into a single Runtime structure.

You might have some luck extracting a generic runner factory of sorts but looking forward to Junit5 I think even that will be done differently by then.

[mpkorstanje]

I see you've just changed the title of the issue. That would be a reasonable request but I don't think the current internal implementation is stable yet.

[mpkorstanje]

If you're looking to contribute something you could start with the develop-v5 branch. The options parsing there is quite stable, and quite repetitive. Should be ideal to construct a builder out of.

[laeubi]

I have started to investigate using StepDefinition from event, the problem is that I can't get access to the actual method, do you think it would be ok to add a getter for the method that belongs to the step definition?