Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Once you're up and running consider the following reference material.
* [Comparing floating point numbers](comparing-floating-point-numbers.md#top)
* [Logging macros](logging.md#top)
* [Test cases and sections](test-cases-and-sections.md#top)
* [Compile time testing](./compile-time-testing.md#top)
* [Test fixtures](test-fixtures.md#top)
* [Explicitly skipping, passing, and failing tests at runtime](skipping-passing-failing.md#top)
* [Reporters (output customization)](reporters.md#top)
Expand Down
153 changes: 153 additions & 0 deletions docs/compile-time-testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
<a id="top"></a>

# Compile-time testing

Catch2 provides a way to test code during compilation. This is
useful for ensuring the code does not lead to undefined behavior.

> [!IMPORTANT]
> This feature is only available with **C++17**

## Why should I test code at compile-time ?

Testing code during compilation is the only way for ensuring the code does
not lead to [undefined behavior](https://en.cppreference.com/w/cpp/language/ub.html).
This is critical to make your program safe, prevent unexpected crashes and worse
consequences of undefined behavior.

## How ?

Catch2 made it easy to test code at compile-time.

The usage is pretty similar to testing at runtime:

| Runtime macro | Compile time macro |
|-----------------|---------------------------|
| `SECTION` | `CONSTEXPR_SECTION` |
| `REQUIRE` | `CONSTEXPR_REQUIRE` |
| `REQUIRE_FALSE` | `CONSTEXPR_REQUIRE_FALSE` |
| `CHECK` | - |
| `CHECK_FALSE` | - |

```c++
TEST_CASE("My compile time test")
{
CONSTEXPR_SECTION("std::array subscript and size")
{
std::array v = {1, 2, 3, 4, 5};
CONSTEXPR_REQUIRE( v.size() == 5 );
CONSTEXPR_REQUIRE( v[2] == 3 );

}; // <-- Don't forget the semicolon here!
}
```

All code inside the `CONSTEXPR_SECTION` will be evaluated during compilation.
If any of the `CONSTEXPR_REQUIRE` or `CONSTEXPR_REQUIRE_FALSE` fails,
it will cause a **compilation error**.

> [!WARNING]
> You cannot use `REQUIRE` or `CHECK` inside a `CONSTEXPR_SECTION`.
> Be careful not to mistake `CONSTEXPR_REQUIRE` with `STATIC_REQUIRE`.
> They both concern compilation-time testing, but they do not have the same
> purpose.

> [!WARNING]
> You cannot nest `CONSTEXPR_SECTION`s or put a `SECTION` inside a
> `CONSTEXPR_SECTION`

> [!NOTE]
> The code inside the `CONSTEXPR_SECTION` is also evaluated at
> runtime. This way, it remains debuggable, and it contributes to the code
> coverage analysis.

## What can I test ?

You can test anything that can be evaluated in a `constexpr` function.
This will depend on your compiler and the C++ standard you are using.
C++20, C++23 and C++26 improved a lot the support for `constexpr`.

Take a look at the C++ compiler support on [cppreference](https://en.cppreference.com/w/cpp/compiler_support.html)
to see what you can use.

## Debug failing compile-time tests

### Investigate from the compiler output

You can check the output of your compiler to find the failing assertion.
The failing line should be highlighted somewhere. If you cannot see any
line:

- Ensure the `CONSTEXPR_SECTION` runs only code that can be evaluated
at compile-time. This will depend on your compiler and the C++ standard you
are using.
- Ensure you did not use unsupported Catch2 macros inside the `CONSTEXPR_SECTION`.
The only supported macros are `CONSTEXPR_REQUIRE` and `CONSTEXPR_REQUIRE_FALSE`.
- Ensure the code you wrote doesn't produce undefined behavior (UB cannot
compile). Reading a range outside its bounds, dereferencing an invalid pointer,
reading a variable after it has been destroyed, are widespread undefined
behaviors. They will be all caught by these tests.

### Investigate at runtime

> [!TIP]
> If you want to debug the code in a `CONSTEXPR_SECTION`, you can simply replace
> the `CONSTEXPR_SECTION` with a `SECTION` and the code will be evaluated at runtime
> only instead. Remember to come back to the `CONSTEXPR_SECTION` once you are
> done !

## Compile time test report

At runtime, a `CONSTEXPR_SECTION` will add a section in the test report. These sections
provide a way to see what was tested during compilation in the test report.

For instance, given the following test case:

```C++
TEST_CASE("My other compile time test")
{
SECTION("Any runtime section")
{
SECTION("Nested section")
{
CONSTEXPR_SECTION("First constexpr section")
{
// ...
};
};

CONSTEXPR_SECTION("Second constexpr section")
{
// ...
};
CONSTEXPR_SECTION("Third constexpr section")
{
// ...
};
CONSTEXPR_SECTION("Fourth section")
{
// ...
};
}

SECTION("Another runtime section")
{
SECTION("Nested runtime section")
{
// ...
}
}
}
```

The report would look like this:

- ✅ My other compile time test
- ✅ Any runtime section
- ✅ Nested section
- ✅ [Passed during compilation] First constexpr section
- ✅ [Passed during compilation] Second constexpr section
- ✅ [Passed during compilation] Third constexpr section
- ✅ [Passed during compilation] Fourth constexpr section
- ✅ Another runtime section
- ✅ Nested runtime section
3 changes: 3 additions & 0 deletions src/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,8 @@ set(IMPL_HEADERS
${SOURCES_DIR}/internal/catch_config_wchar.hpp
${SOURCES_DIR}/internal/catch_console_colour.hpp
${SOURCES_DIR}/internal/catch_console_width.hpp
${SOURCES_DIR}/internal/catch_constexpr_asserts.hpp
${SOURCES_DIR}/internal/catch_constexpr_section.hpp
${SOURCES_DIR}/internal/catch_container_nonmembers.hpp
${SOURCES_DIR}/internal/catch_context.hpp
${SOURCES_DIR}/internal/catch_debug_console.hpp
Expand Down Expand Up @@ -173,6 +175,7 @@ set(IMPL_SOURCES
${SOURCES_DIR}/internal/catch_clara.cpp
${SOURCES_DIR}/internal/catch_commandline.cpp
${SOURCES_DIR}/internal/catch_console_colour.cpp
${SOURCES_DIR}/internal/catch_constexpr_section.cpp
${SOURCES_DIR}/internal/catch_context.cpp
${SOURCES_DIR}/internal/catch_debug_console.cpp
${SOURCES_DIR}/internal/catch_debugger.cpp
Expand Down
2 changes: 2 additions & 0 deletions src/catch2/catch_all.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@
#include <catch2/internal/catch_config_wchar.hpp>
#include <catch2/internal/catch_console_colour.hpp>
#include <catch2/internal/catch_console_width.hpp>
#include <catch2/internal/catch_constexpr_asserts.hpp>
#include <catch2/internal/catch_constexpr_section.hpp>
#include <catch2/internal/catch_container_nonmembers.hpp>
#include <catch2/internal/catch_context.hpp>
#include <catch2/internal/catch_debug_console.hpp>
Expand Down
20 changes: 18 additions & 2 deletions src/catch2/catch_test_macros.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,16 @@
#ifndef CATCH_TEST_MACROS_HPP_INCLUDED
#define CATCH_TEST_MACROS_HPP_INCLUDED

#include <catch2/internal/catch_test_macro_impl.hpp>
#include <catch2/catch_message.hpp>
#include <catch2/catch_user_config.hpp>
#include <catch2/internal/catch_constexpr_asserts.hpp>
#include <catch2/internal/catch_constexpr_section.hpp>
#include <catch2/internal/catch_section.hpp>
#include <catch2/internal/catch_test_macro_impl.hpp>
#include <catch2/internal/catch_test_registry.hpp>
#include <catch2/internal/catch_unique_name.hpp>
#include <catch2/internal/catch_unreachable.hpp>


// All of our user-facing macros support configuration toggle, that
// forces them to be defined prefixed with CATCH_. We also like to
// support another toggle that can minimize (disable) their implementation.
Expand Down Expand Up @@ -48,6 +49,7 @@
#define CATCH_REGISTER_TEST_CASE( Function, ... ) INTERNAL_CATCH_REGISTER_TESTCASE( Function, __VA_ARGS__ )
#define CATCH_SECTION( ... ) INTERNAL_CATCH_SECTION( __VA_ARGS__ )
#define CATCH_DYNAMIC_SECTION( ... ) INTERNAL_CATCH_DYNAMIC_SECTION( __VA_ARGS__ )
#define CATCH_CONSTEXPR_SECTION( ... ) INTERNAL_CATCH_CONSTEXPR_SECTION( __VA_ARGS__ )
#define CATCH_FAIL( ... ) do { \
INTERNAL_CATCH_MSG("CATCH_FAIL", Catch::ResultWas::ExplicitFailure, Catch::ResultDisposition::Normal, __VA_ARGS__ ); \
Catch::Detail::Unreachable(); \
Expand All @@ -72,6 +74,8 @@
#define CATCH_STATIC_CHECK_FALSE( ... ) CATCH_CHECK_FALSE( __VA_ARGS__ )
#endif

#define CATCH_CONSTEXPR_REQUIRE( ... ) INTERNAL_CATCH_CONSTEXPR_ASSERT( "CONSTEXPR_REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__ )
#define CATCH_CONSTEXPR_REQUIRE_FALSE( ... ) INTERNAL_CATCH_CONSTEXPR_ASSERT( "CONSTEXPR_REQUIRE_FALSE", Catch::ResultDisposition::Normal | Catch::ResultDisposition::FalseTest, __VA_ARGS__ )

// "BDD-style" convenience wrappers
#define CATCH_SCENARIO( ... ) CATCH_TEST_CASE( "Scenario: " __VA_ARGS__ )
Expand Down Expand Up @@ -109,6 +113,7 @@
#define CATCH_REGISTER_TEST_CASE( Function, ... ) (void)(0)
#define CATCH_SECTION( ... )
#define CATCH_DYNAMIC_SECTION( ... )
#define CATCH_CONSTEXPR_SECTION( ... )
#define CATCH_FAIL( ... ) (void)(0)
#define CATCH_FAIL_CHECK( ... ) (void)(0)
#define CATCH_SUCCEED( ... ) (void)(0)
Expand All @@ -119,6 +124,9 @@
#define CATCH_STATIC_CHECK( ... ) (void)(0)
#define CATCH_STATIC_CHECK_FALSE( ... ) (void)(0)

#define CATCH_CONSTEXPR_REQUIRE( ... ) (void)(0)
#define CATCH_CONSTEXPR_REQUIRE_FALSE( ... ) (void)(0)

// "BDD-style" convenience wrappers
#define CATCH_SCENARIO( ... ) INTERNAL_CATCH_TESTCASE_NO_REGISTRATION(INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEST_ ))
#define CATCH_SCENARIO_METHOD( className, ... ) INTERNAL_CATCH_TESTCASE_METHOD_NO_REGISTRATION(INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEST_ ), className )
Expand Down Expand Up @@ -155,6 +163,8 @@
#define REGISTER_TEST_CASE( Function, ... ) INTERNAL_CATCH_REGISTER_TESTCASE( Function, __VA_ARGS__ )
#define SECTION( ... ) INTERNAL_CATCH_SECTION( __VA_ARGS__ )
#define DYNAMIC_SECTION( ... ) INTERNAL_CATCH_DYNAMIC_SECTION( __VA_ARGS__ )
#define CONSTEXPR_SECTION( ... ) INTERNAL_CATCH_CONSTEXPR_SECTION( __VA_ARGS__ )
#define SILENT_CONSTEXPR_SECTION( ... ) INTERNAL_CATCH_SILENT_CONSTEXPR_SECTION( __VA_ARGS__ )
#define FAIL( ... ) do { \
INTERNAL_CATCH_MSG( "FAIL", Catch::ResultWas::ExplicitFailure, Catch::ResultDisposition::Normal, __VA_ARGS__ ); \
Catch::Detail::Unreachable(); \
Expand All @@ -179,6 +189,9 @@
#define STATIC_CHECK_FALSE( ... ) CHECK_FALSE( __VA_ARGS__ )
#endif

#define CONSTEXPR_REQUIRE( ... ) INTERNAL_CATCH_CONSTEXPR_ASSERT( "CONSTEXPR_REQUIRE", Catch::ResultDisposition::Normal, __VA_ARGS__ )
#define CONSTEXPR_REQUIRE_FALSE( ... ) INTERNAL_CATCH_CONSTEXPR_ASSERT( "CONSTEXPR_REQUIRE_FALSE", Catch::ResultDisposition::Normal | Catch::ResultDisposition::FalseTest, __VA_ARGS__ )

// "BDD-style" convenience wrappers
#define SCENARIO( ... ) TEST_CASE( "Scenario: " __VA_ARGS__ )
#define SCENARIO_METHOD( className, ... ) INTERNAL_CATCH_TEST_CASE_METHOD( className, "Scenario: " __VA_ARGS__ )
Expand Down Expand Up @@ -215,6 +228,7 @@
#define REGISTER_TEST_CASE( Function, ... ) (void)(0)
#define SECTION( ... )
#define DYNAMIC_SECTION( ... )
#define CONSTEXPR_SECTION( ... )
#define FAIL( ... ) (void)(0)
#define FAIL_CHECK( ... ) (void)(0)
#define SUCCEED( ... ) (void)(0)
Expand All @@ -224,6 +238,8 @@
#define STATIC_REQUIRE_FALSE( ... ) (void)(0)
#define STATIC_CHECK( ... ) (void)(0)
#define STATIC_CHECK_FALSE( ... ) (void)(0)
#define CONSTEXPR_REQUIRE( ... ) (void)(0)
#define CONSTEXPR_REQUIRE_FALSE( ... ) (void)(0)

// "BDD-style" convenience wrappers
#define SCENARIO( ... ) INTERNAL_CATCH_TESTCASE_NO_REGISTRATION(INTERNAL_CATCH_UNIQUE_NAME( CATCH2_INTERNAL_TEST_ ) )
Expand Down
43 changes: 43 additions & 0 deletions src/catch2/internal/catch_constexpr_asserts.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@

// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)

// SPDX-License-Identifier: BSL-1.0
#ifndef CATCH_CONSTEXPR_ASSERTS_HPP_INCLUDED
#define CATCH_CONSTEXPR_ASSERTS_HPP_INCLUDED
#include <catch2/internal/catch_decomposer.hpp>
#include <catch2/internal/catch_test_macro_impl.hpp>

namespace Catch {
template <typename ExpressionType>
void handleExpression( StringRef macroName,
SourceLineInfo const& lineInfo,
StringRef capturedExpression,
ResultDisposition::Flags resultDisposition,
ExpressionType decomposedExpression ) {
AssertionHandler catchAssertionHandler(
macroName, lineInfo, capturedExpression, resultDisposition );
catchAssertionHandler.handleExpr( decomposedExpression );
catchAssertionHandler.complete();
}
} // namespace Catch

#define INTERNAL_CATCH_CONSTEXPR_ASSERT( macroName, resultDisposition, ... ) \
do { \
CATCH_INTERNAL_START_WARNINGS_SUPPRESSION \
CATCH_INTERNAL_SUPPRESS_PARENTHESES_WARNINGS \
const auto expr__InternalCatch__ = Catch::Decomposer() <= __VA_ARGS__; \
CATCH_INTERNAL_STOP_WARNINGS_SUPPRESSION \
if ( expr__InternalCatch__.getResult() != \
Catch::isFalseTest( resultDisposition ) ) { \
} else \
Catch::handleExpression( macroName##_catch_sr, \
CATCH_INTERNAL_LINEINFO, \
CATCH_INTERNAL_STRINGIFY( __VA_ARGS__ ), \
resultDisposition, \
expr__InternalCatch__ ); \
} while ( false )

#endif // CATCH_CONSTEXPR_ASSERTS_HPP_INCLUDED
23 changes: 23 additions & 0 deletions src/catch2/internal/catch_constexpr_section.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@

// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)

// SPDX-License-Identifier: BSL-1.0

#include <catch2/internal/catch_assertion_handler.hpp>
#include <catch2/internal/catch_constexpr_section.hpp>

namespace Catch {
void ConstexprSection::addFakeAssertion() {
AssertionHandler catchAssertionHandler(
"CONSTEXPR_SECTION",
SourceLineInfo( "unknown-file", 0 ),
"[Passed during compilation]",
ResultDisposition::Normal );
catchAssertionHandler.handleMessage( ResultWas::Ok,
"[Passed during compilation]" );
catchAssertionHandler.complete();
}
} // namespace Catch
49 changes: 49 additions & 0 deletions src/catch2/internal/catch_constexpr_section.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@

// Copyright Catch2 Authors
// Distributed under the Boost Software License, Version 1.0.
// (See accompanying file LICENSE.txt or copy at
// https://www.boost.org/LICENSE_1_0.txt)

// SPDX-License-Identifier: BSL-1.0
#ifndef CATCH_CONSTEXPR_SECTION_HPP_INCLUDED
#define CATCH_CONSTEXPR_SECTION_HPP_INCLUDED

#if defined( CATCH_CPP17_OR_GREATER ) && __cpp_constexpr >= 201603
# define CATCH_CONFIG_CONSTEXPR_SECTIONS
#endif

#include <catch2/internal/catch_compiler_capabilities.hpp>
#include <catch2/internal/catch_section.hpp>
#include <catch2/internal/catch_unique_name.hpp>

namespace Catch {

struct ConstexprSection {
template <typename Callable>
ConstexprSection( Callable callable ) {
static_assert( ( callable(), true ), "CONSTEXPR_SECTION failure" );
addFakeAssertion();
callable();
}

// Add a fake assertion to the section to avoid failures because the
// section is empty
static void addFakeAssertion();
};
} // namespace Catch

#ifdef CATCH_CONFIG_CONSTEXPR_SECTIONS

# define INTERNAL_CATCH_CONSTEXPR_SECTION( ... ) \
INTERNAL_CATCH_SECTION( "[Passed during compilation] " __VA_ARGS__ ) \
[[maybe_unused]] const ::Catch::ConstexprSection \
INTERNAL_CATCH_UNIQUE_NAME( \
catch_internal_CompileTimeSection ) = [&]()

#else
# define INTERNAL_CATCH_CONSTEXPR_SECTION( ... ) \
static_assert( false, "C++17 is required for CONSTEXPR_SECTION" ); \
if ( false )
#endif

#endif // CATCH_CONSTEXPR_SECTION_HPP_INCLUDED
Loading
Loading