[Suggestion] Allow Method-Level Parameters

[andcadev]

Description

If I’m not missing something, the LUnit Parameterized Add-On allows defining test classes with variable parameters, enabling theory-like tests.
Currently, parameters are applied at the class level, meaning that the same parameter set is shared by all test methods in the class. This results in test methods being executed multiple times even when a specific method does not actually use a parameter-based approach.

In most modern unit testing frameworks (e.g. JUnit, NUnit, xUnit), parameterized or theory-style tests associate parameter sets with individual test methods rather than with the entire test class.

Problem

Applying parameters only at the class level introduces several limitations:

• All test methods are forced to share the same parameters, even when their test logic is unrelated.
• Test classes tend to grow artificially large or fragmented to accommodate different parameter sets.
• It reduces flexibility compared to theory-based testing approaches found in other frameworks.

Expected Behavior

It should be possible to define variable parameters at the method level, so that each test method can:

• Declare its own parameter set.
• Execute independently from other methods in the same class.
• Better reflect the intent of theory-style testing.

Possible approaches could include:

• Allowing parameter definitions to be associated with individual test methods.
• Supporting both class-level and method-level parameters

Motivation

Supporting method-level parameters would:

• Improve test clarity and intent.
• Reduce unnecessary class duplication or fragmentation.
• Make the Parameterized Add-On more expressive and closer to modern theory-based unit testing frameworks.

[antsundq]

Hi and thanks for another well written suggestion.

Could you please help me understand the problem this would solve? Maybe you can share an example which illustrates it, as I do not perceive this problem in the way I write tests.

My view on the points you bring up:

• All test methods are forced to share the same parameters, even when their test logic is unrelated.

This is actually intended. If the tests are not sharing parameters they do not belong in the same teste case class. This might be personal preference, but I do not see it as a bad thing to have many small classes with cohesive tests. I do not view it as bad in any way to have e.g. a parametrized test with a single test method.

• Test classes tend to grow artificially large or fragmented to accommodate different parameter sets.

This I do not get? Do you mean that you would have some parameters used by a subset of methods and some other parameters used by other methods? My suggestion would be to split the parameterized tests into smaller test classes.

• It reduces flexibility compared to theory-based testing approaches found in other frameworks.

By this you mean flexibility of having different parameter sets for different test methods? So it gets down to a trade of between coupling and copying? If we have by-method parameters we may need to maintain copies of the same parameters between the tests which needs the same parameters. It would also seem to me that having both class and method level parameters would add so much complexity that the readability would be impaired.

Do keep in mind that implementing something similar to annotations containing parameters and calling the methods using these parameters as arguments, would be very messy in LabVIEW. I do not see any direct way to implement by-method parameters which would be better than just looping over an array of parameters in a normal test case. But that does of course not mean that there isn't any, so let me know if you have any ideas on what this would look like.

[andcadev]

Maybe you can share an example which illustrates it, as I do not perceive this problem in the way I write tests

My approach is slightly different. Typically, I create one test class for each class I want to test.
Imagine a Calculator class with 2 public methods: Add() and Subtract(). I will create the CalculatorTest class with N methods for testing add and M methods for testing subtract.
Of course, with this approach, test methods for the Add method cannot use the same parameters used for the Subtract method.
That's why I asked for method-level parameters.

---

This is actually intended. If the tests are not sharing parameters they do not belong in the same teste case class.

We are using 2 different approaches. Given your design philosophy, I understand why method-level parameters may not fit the current approach. I’m fine with closing this issue.

---

By this you mean flexibility of having different parameter sets for different test methods? So it gets down to a trade of between coupling and copying? If we have by-method parameters we may need to maintain copies of the same parameters between the tests which needs the same parameters.

In my view, when it comes to unit testing, I value flexibility over raw performance. If the framework consumes a reasonable amount of additional memory or takes slightly longer to execute but allows different user approaches, I consider that acceptable. For example, if running my test cases takes 5 minutes instead of 4.5 and uses 500 MB of memory instead of 400 MB, that’s perfectly fine. Of course, there is a limit — performance should remain within practical boundaries

---

Do keep in mind that implementing something similar to annotations containing parameters and calling the methods using these parameters as arguments, would be very messy in LabVIEW. I do not see any direct way to implement by-method parameters which would be better than just looping over an array of parameters in a normal test case. But that does of course not mean that there isn't any, so let me know if you have any ideas on what this would look like.

Actually I've been thinking about creating a toolkit for writing and reading annotations to classes and methods for a while. Maybe it's time to start working on that :)
For now, the only way that comes to my mind is building a registration mechanism where methods register themeselves to a specific parameter cluster and value set but I discarded it because it’s too complex and puts too much responsibility on the user.

It’s a great framework, and I hope it becomes more widely adopted over time.

[antsundq]

Okay, I see. It is then down to how you define the "unit" in unit testing. I am not as strict with keeping the classes 1:1 with test case classes.

Regarding the performance bit, this is something I really care about since I mostly work in a TDD style and run my tests a lot during development. For me, if the tests take more than a second or so to run, it brakes my flow, and this is why I care about shaving those seconds off. 5 minutes to run unit tests would be totally unacceptable. As an example, for the current project I am working on I have 686 tests and it takes about 10 seconds to run the full test suite. This is already so long that I rarely have patience to run the full test suite, except as part of CI.

Sounds like you could have a good use case for annotations here. Once you have that, it should be quite simple to create an add-on to use it for parameterizing tests.

[antsundq]

It’s a great framework, and I hope it becomes more widely adopted over time.

And thanks for this comment, great to hear. :)