Skip to content

Update Spack lecture, demo notes, and exercise #206

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
IshaanDesai merged 4 commits into from
Nov 26, 2024
Merged

Update Spack lecture, demo notes, and exercise #206

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
88ffcc5
Update Spack version and installation commands
IshaanDesai Nov 25, 2024
77fad0f
Add notes for installing curl manually to the Spack demo
IshaanDesai Nov 25, 2024
0c9d839
Update Spack demo notes
IshaanDesai Nov 26, 2024
4619f8b
Update Spack exercise
IshaanDesai Nov 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 27 additions & 18 deletions 03_building_and_packaging/spack_demo.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@
- Git repository of Python scripts

```bash
git clone -b releases/v0.19 https://github.com/spack/spack.git
git clone -b v0.23.0 -c feature.manyFiles=true --depth=2 https://github.com/spack/spack.git
```

- `v0.19` is currently the latest release
- Update to newer versions using `git pull`/`git checkout -b`
- `v0.23.0` is currently the latest major release

- **Note:** Install `curl`, `libcurl4-openssl-dev`, and `vim` to ensure that Spack v0.23.0 works in a fresh Ubuntu Jammy container.

- Initializing Spack with

Expand Down Expand Up @@ -159,6 +160,14 @@

We see that `cmake` is an implicit dependency as we need it for building our package.

- If the same Docker container as in step 2 is used, make sure to uninstall `zlib` before installing `helloworld`.

- Make sure Spack finds external packages that `HelloWorld` needs

```bash
spack external find
```

- Install package

```bash
Expand Down Expand Up @@ -187,29 +196,29 @@

This will concretize (internally, i.e. no output on terminal) and then build the software.

- If one wants to edit the package later, there are two options

```bash
spack edit helloworld
```

or open `package.py` file in `${HOME}/var/spack/repos/builtin/packages/helloworld/`

- **Optional**: one could add `main` branch and thus GitHub repository

```diff
+ git = "https://github.com/Simulation-Software-Engineering/HelloWorld.git"
+
+ version('main', branch='main')
+ version("main", branch="main")
```

This can also be used for `develop` branches etc. It is useful if one needs really the newest version of a package or if one develops software using Spack.

- If one wants to edit the package later, there are two options

```bash
spack edit helloworld
```

or open `package.py` file in `${HOME}/var/spack/repos/builtin/packages/helloworld/`

- Add artificial dependencies

```diff
+ depends_on('python@3:', when='@0.3.0:')
+ depends_on('zlib@:1.2')
+ depends_on("python@3:", when="@0.3.0:")
+ depends_on("zlib@:1.2")
```

This means that the package depends on Python `3.0.0` or newer and newer if we use `helloworld` of version `0.3.0` or newer. The software also requires at most `zlib` in version `1.2.10`
Expand All @@ -227,9 +236,9 @@
- Add an artificial variant

```diff
+ variant('python', default=True, description='Enable Python support')
- depends_on('python@3:', when='@0.3.0:')
+ depends_on('python@3:', when='@0.3.0:+python')
- depends_on("python@3:", when="@0.3.0:")
+ variant("python", default=True, description="Enable Python support")
+ depends_on("python@3:", when="@0.3.0:+python")
```

and check its existence
Expand All @@ -246,7 +255,7 @@

`~` can be (often) used instead of `-`. There are [examples in the documentation](https://spack.readthedocs.io/en/latest/basic_usage.html#variants).

## Further reading
## Further material

### References

Expand Down
6 changes: 3 additions & 3 deletions 03_building_and_packaging/spack_exercise.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ This exercise is about packaging code with Spack. We work with a simplified vers

At the end of the exercise you find a section with hints and remarks. Make sure to check this section.

Deadline: **Thursday, December 15th, 2022, 9:00**
Deadline: **Wednesday, December 4th, 2024, 09:00**

## Creation of a Spack Package

We want to package the [Spack example code on GitHub](https://github.com/Simulation-Software-Engineering/spack-exercise). It is an adapted version of the previous week's exercise again. The `deal.ii` dependency has been removed as compiling `deal.ii` from scratch takes too long. Additionally, several releases with different dependencies have been added. The code in the repository creates an executable that is called `spackexample`. It also creates a library `libspackexamplelib` (what a name) which is needed to run `spackexample`.
We want to package the [Spack example code on GitHub](https://github.com/Simulation-Software-Engineering/spack-exercise). It is an adapted version of the last week's exercise. The `deal.ii` dependency has been removed as compiling `deal.ii` from scratch takes too long. Additionally, several releases with different dependencies have been added. The code in the repository creates an executable that is called `spackexample`. It also creates a library `libspackexamplelib` (what a name) which is needed to run `spackexample`.

The default name of your Spack package is `spack-exercise`. The Spack package should create the executable `spackexample` and the corresponding library `libspackexamplelib` mentioned above. Under normal circumstances you do not need to edit the CMake configuration for this exercise.

Expand All @@ -24,7 +24,7 @@ Create a Spack package for all releases of the given code and make sure that the

### Development/Packaging Environment

Please do all the development inside a Docker container. The Docker container is based on the image built from the recipe provided in the [exercise repository](https://github.com/Simulation-Software-Engineering/spack-exercise). You can find the recipe of the image inside the `docker/` directory of [the example code on GitHub](https://github.com/Simulation-Software-Engineering/spack-exercise). The image itself is based on Ubuntu 20.04 and has the Boost dependency preinstalled. Additionally, Spack has been set up in the recipe. Two editors `vim` and `nano` preinstalled. If you want to install further software in your container you are free to do so.
Please do all the development inside a Docker container. The Docker container is based on the image built from the recipe provided in the [exercise repository](https://github.com/Simulation-Software-Engineering/spack-exercise). You can find the recipe of the image inside the `docker/` directory. The image itself is based on Ubuntu 20.04 and has the Boost dependency preinstalled. Additionally, Spack has been set up in the recipe. Two editors `vim` and `nano` are preinstalled. If you want to install further software in your container you are free to do so.

### Packaging Steps

Expand Down
15 changes: 7 additions & 8 deletions 03_building_and_packaging/spack_slides.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,9 @@ slideOptions:

## Learning goals

- What are the challenges when bringing (your) software to supercomputers?
- How to use Spack to install software.
- How to create a Spack package for your own software.
- Understand the challenges when bringing (your) software to supercomputers.
- Use Spack to install software.
- Create a Spack package for your own software.
Comment on lines -41 to +43
Copy link
Member Author

@IshaanDesai IshaanDesai Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@uekerman learning goals okay?

Copy link
Member

@uekerman uekerman Nov 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better be more specific here. "Understand" is in general discouraged as too unspecific. Also in the other two points, you could be more specific. Should students be able to create Spack packages for any software?

Copy link
Member Author

@IshaanDesai IshaanDesai Nov 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should students be able to create Spack packages for any software?

In principle yes. We show them how to create the package in a reasonably generic way.

I will be more specific with the learning goals and directly update on the main branch.


---

Expand Down Expand Up @@ -104,24 +104,23 @@ slideOptions:
- Steps:
1. Install and configure Spack
2. Learn how to use Spack for package management and installation
3. Create a Spack package for our HelloWorld code
3. Create a Spack package for a HelloWorld code
- Code is on [GitHub](https://github.com/Simulation-Software-Engineering/HelloWorld)

---

## Spack Installation

- Dependencies: Python, Git, C/C++ compiler, patch, make, tar...
- Basically `build-essentials`, `git`, and `python` on Ubuntu
- Basically `build-essential`, `git`, and `python` on Ubuntu
- May be old, install newer versions with Spack if needed
- Installation in user-writable location

```bash
git clone -b releases/v0.19 https://github.com/spack/spack.git
git clone -b v0.23.0 -c feature.manyFiles=true --depth=2 https://github.com/spack/spack.git
```

- `v0.19` is currently the latest release
- Update to newer versions using `git pull`/`git checkout -b`
- `v0.23.0` is currently the latest major release

---

Expand Down