Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#17754: Lower Indestructible to Metal, add guidance on using static vars with non-trivial destructors #17899

Merged
merged 3 commits into from
Feb 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ tt_metal/hw/firmware/src/*erisc* @aliuTT @ubcheema
tt_metal/hw/inc/ethernet/ @aliuTT @ubcheema
tt_metal/hw/inc/wormhole/eth_l1_address_map.h @aliuTT @ubcheema
tt_metal/third_party/tt_llk_* @rtawfik01 @ttmtrajkovic @rdjogoTT
tt_metal/tt_stl/ @patrickroberts @ayerofieiev-tt @dmakoviichuk-tt @sminakov-tt
tt_metal/tt_stl/ @patrickroberts @ayerofieiev-tt @dmakoviichuk-tt @sminakov-tt @omilyutin-tt

sfpi/ @pgkeller

Expand Down
55 changes: 53 additions & 2 deletions contributing/BestPractices.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Best Practices for C++20 Repository
# Best Practices for Contributing to TT Metal

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropping "C++20" part. This is misleading in some cases, for example, the doc suggests to use tt:stl::Span while C++20 has std::span

## 1. Pass Complex Types by Const References

Expand Down Expand Up @@ -319,7 +319,7 @@ struct PadDimension {
```
Motivation
- **Bug Prevention:** Reduces the risk of bugs due to uninitialized variables.
- **Code Safety:** Ensures that all variables have a known value, leading to safer and more predictable code.
- **Safety:** Ensures that all variables have a known value, leading to safer and more predictable code.
- **Ease of Review:** Simplifies code reviews by making initialization explicit.

## 16. Use Early Exit for Contract Checks
Expand Down Expand Up @@ -354,3 +354,54 @@ void doSomething(...) {
- **Code Clarity:** Improves code clarity by reducing unnecessary nesting.
- **Maintainability:** Makes the code easier to maintain by focusing on the main logic once preconditions are validated.
- **Efficiency:** Potentially improves performance by avoiding unnecessary processing when contract conditions aren't met.

## 17. Avoid `static` variables with non-trivial destructors
### Practice
Avoid using `static` variables with non-trivial destructors. When applicable, use `tt::stl::Indestructible<T>` to create static objects with disabled destructor.

### Explanation
Objects with static storage duration (globals, static class members, or function-local statics) live from initialization until program termination.

A non-trivial destructor (i.e., one that is user-defined or virtual) may depend on the state of other objects, which might have already been destroyed by the time it is invoked. This can lead to undefined behavior or subtle bugs, especially in the multi-threaded environments.

An object is considered trivially destructible if it has no custom or virtual destructor and all its bases and non-static members are also trivially destructible. Examples include: fundamental types (pointers, int, float, etc.), arrays of trivially destructible types, variables marked with `constexpr`.

To ensure safe and predictable program termination, static objects should meet these criteria. If dynamic initialization is required, consider using function-local statics with `tt::stl::Indestructible<T>` that disables destruction.

### Motivation
- **Safety:** Prevents accessing objects after they have been destroyed.
- **Maintainability:** Simplifies tracking the lifetime of objects and helps avoid errors related to destruction ordering.

### Example
**Avoid:**
```cpp
// Bad: Using a static object with a non-trivial destructor.
static const std::map<int, std::string> kDeviceConfigFiles = {
{1, "n150.yaml"},
{2, "n300.yaml"},
{8, "t3000.yaml"}
};
```

Comment on lines +378 to +384
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What can go wrong in this example? Is it better to include an example with custom destructor like device?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you retain references to either key/value in another static singleton, that other destructor might access it after the memory is freed. This is of course overly simplistic and in isolation looks unrealistic, but just to illustrate the point. Easier to always be safe than to think of when / how things might break.

**Prefer:**
```cpp
// Option 1: Use a trivial type for static data when possible.
constexpr std::string_view kData = "Trivial destructor! Good!";

constexpr uint32_t kMaxNumberOfCommandQueues = 2;

// Using array of trivially destructible types is OK.
constexpr std::array<int, 3> kDeviceIds = {1, 2, 8};

// Option 2: If dynamic initialization is required, use function-local statics with `Indestructible`.
const auto& get_device_configs() {
static tt::stl::Indestructible<std::map<int, std::string_view>> configs{
std::map<int, std::string_view>{
{1, "n150.yaml"},
{2, "n300.yaml"},
{8, "t3000.yaml"}
}
};
return configs.get();
}
```
1 change: 1 addition & 0 deletions tests/tt_metal/tt_metal/stl/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
set(UNIT_TESTS_STL_SRC
${CMAKE_CURRENT_SOURCE_DIR}/test_any_range.cpp
${CMAKE_CURRENT_SOURCE_DIR}/test_indestructible.cpp
${CMAKE_CURRENT_SOURCE_DIR}/test_slotmap.cpp
${CMAKE_CURRENT_SOURCE_DIR}/test_strong_type.cpp
)
Expand Down
25 changes: 25 additions & 0 deletions tests/tt_metal/tt_metal/stl/test_indestructible.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// SPDX-FileCopyrightText: © 2024 Tenstorrent Inc.
//
// SPDX-License-Identifier: Apache-2.0

#include <gtest/gtest.h>
#include <gmock/gmock.h>

#include "tt_metal/tt_stl/indestructible.hpp"

namespace tt::stl {
namespace {

TEST(IndestructibleTest, Basic) {
struct DangerouslyDestructible {
~DangerouslyDestructible() {
// Wrapping in a lambda, as `FAIL()` returns `void`.
[]() { FAIL(); }();
}
};

Indestructible<DangerouslyDestructible> obj;
}

} // namespace
} // namespace tt::stl
2 changes: 1 addition & 1 deletion tt-train/sources/ttml/autograd/auto_context.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ uint32_t AutoContext::get_seed() const {
}

AutoContext& AutoContext::get_instance() {
static core::Indestructible<AutoContext> instance{};
static tt::stl::Indestructible<AutoContext> instance{};
return instance.get();
}
std::optional<NodeId> AutoContext::add_backward_node(GradFunction&& grad_function, std::span<NodeId> links) {
Expand Down
4 changes: 2 additions & 2 deletions tt-train/sources/ttml/autograd/auto_context.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@

#pragma once

#include <indestructible.hpp>
#include <memory>
#include <random>

#include "core/indestructible.hpp"
#include "core/mesh_device.hpp"
#include "graph.hpp"

Expand Down Expand Up @@ -62,7 +62,7 @@ class AutoContext {
tt::tt_metal::distributed::MeshShape m_mesh_shape = {1, 1};
std::unique_ptr<core::MeshDevice> m_device;

friend class core::Indestructible<AutoContext>;
friend class tt::stl::Indestructible<AutoContext>;
};

inline auto& ctx() {
Expand Down
40 changes: 0 additions & 40 deletions tt-train/sources/ttml/core/indestructible.hpp

This file was deleted.

51 changes: 51 additions & 0 deletions tt_metal/tt_stl/indestructible.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// SPDX-FileCopyrightText: (c) 2024 Tenstorrent AI ULC
//
// SPDX-License-Identifier: Apache-2.0

#pragma once

#include <cstddef>
#include <new>
#include <utility>

namespace tt::stl {

// `Indestructible` is a wrapper around `T` that behaves like `T` but does not call the destructor of `T`.
Copy link
Collaborator

@cfjchu cfjchu Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While we're deliberately not calling destructor forT, doesn't this break RAII gurantees if T holds resources like file handles or mutexes and ~T() was supposed to execute some cleanup subroutine?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats the point:) Static vars shouldn't be responsible for this, as the stuff they are cleaning up (or other dependencies that are needed for this) might be gone already.

// This is useful for creating objects with static storage duration: `Indestructible` avoids heap allocation, provides
// thread-safe construction, and ensures the destructor is no-op, so does not depend on any other objects.
//
//
// Example usage:
//
// const auto& get_object() {
// static Indestructible<MyObject> object;
// return object.get();
// }
//
template <typename T>
class Indestructible {
public:
template <typename... Args>
explicit Indestructible(Args&&... args) {
// Construct T in our aligned storage
new (&storage_) T(std::forward<Args>(args)...);
}

T& get() { return *std::launder(reinterpret_cast<T*>(&storage_)); }

const T& get() const { return *std::launder(reinterpret_cast<const T*>(&storage_)); }

// Disable copy and assignment
Indestructible(const Indestructible&) = delete;
Indestructible& operator=(const Indestructible&) = delete;

// Destructor does NOT call T's destructor.
// This leaves the object "indestructible."
~Indestructible() = default;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the moment we start leveraging LeakSanitizer we're going to get flooded with reports?

Why can't we just enforce proper scopes and avoid globals/leaks/UB?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIK asan and leak sanitizers would be happy about it - as memory remains reachable at program exit, while we never run into use-after-free because all dtors are disabled. Having said that, I don't know for sure, but I would be very surprised if sanitizers don't understand this very common pattern.

Why can't we just enforce proper scopes and avoid globals/leaks/UB?

Sometimes it is way easier to use this pattern, basically wherever singleton pattern actually applies, but safer and more efficient.


private:
// A buffer of std::byte with alignment of T and size of T
alignas(T) std::byte storage_[sizeof(T)];
};

} // namespace tt::stl
Loading