Skip to content

Commit

Permalink
add handing variables in github actions
Browse files Browse the repository at this point in the history
  • Loading branch information
SteveSayantan committed Jan 21, 2025
1 parent e3d776b commit 6f3a8c7
Show file tree
Hide file tree
Showing 5 changed files with 315 additions and 1 deletion.
138 changes: 138 additions & 0 deletions .github/chainingJobs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
## Idea
In a workflow, we generally have multiple jobs. By default, each of the jobs runs parallelly in separate runners given enough runners are available. If sufficient runners are not available, some of the jobs will be queued until a runner is free.

If we use a GitHub-hosted runner, each job runs in a **fresh instance** of a runner image specified by runs-on.

However, the order in which the jobs run can be controlled.

- #### Parallel Execution
Here, job1 and job2 will execute simultaneously in two separate runners (default behavior).

```yaml
name: Parallel Jobs

on: workflow_dispatch
jobs:
job1:
runs-on: ubuntu-latest
steps:
- run: echo "Job 1"
job2:
runs-on: ubuntu-latest
steps:
- run: echo "Job 2"

```

- #### Sequential Execution with Dependencies
Jobs can depend on other jobs using the `needs` keyword. This creates a dependency chain and ensures that dependent jobs execute only after their prerequisites are completed.

If any of the prerequisites fails or skips, the corresponding job using `needs` skips.

```yaml
name: Dependent Jobs

on: workflow_dispatch

jobs:
build:
runs-on: ubuntu-latest
steps:
- run: echo "Building project"

test:
runs-on: ubuntu-latest
needs: build # job with id `build` must execute successfully to run this job
steps:
- run: echo "Running tests"

deploy:
runs-on: ubuntu-latest
needs: test # we can specify multiple dependencies in an array
steps:
- run: echo "Deploying"

```
- #### Conditional Execution
Jobs can have conditional execution rules using the `if` keyword. We can use it to prevent a step from running unless a condition is met.

```yaml
name: Conditional Jobs
on: workflow_dispatch
jobs:
build:
runs-on: ubuntu-latest
steps:
- run: echo "Building project"

deploy:
runs-on: ubuntu-latest
needs: build
if: github.ref == 'refs/heads/main' # this job runs only if the current branch is main
steps:
- run: echo "Deploying to production"

```

## Example

```yaml
name: Chaining Jobs

on:
workflow_dispatch:

inputs:
run-job-3:
description: "Run job 3"
type: boolean

jobs:

job-1:
name: Job 1
runs-on: ubuntu-latest
steps:
- name: Output for Job 1
run: echo "Hello from Job 1. Run Job 3 equals ${{ github.event.inputs.run-job-3 }}"

job-2:
name: Job 2
runs-on: ubuntu-latest
needs:
- job-1
steps:
- name: Output for Job 2
run: echo "Hello from Job 2"

job-3:
name: Job 3
if: github.event.inputs.run-job-3 == 'true' # if the input is false, this job would be skipped
runs-on: ubuntu-latest
needs:
- job-1
steps:
- name: Output for Job 3
run: echo "Hello from Job 3"

job-4:
name: Job 4
runs-on: ubuntu-latest
# if: always() # Causes the job to always execute
needs:
- job-2
- job-3
steps:
- name: Output for Job 4
run: echo "Hello from Job 4"

```
#### Explanation
- Here, every job runs on a separate fresh runner image.
- `job-1` runs first.

- `job-2` and `job-3` depends on `job-1`. They run after `job-1` finishes successfully. `job-3` also depends on the input condition.

- `job-4` depends on both `job-2` and `job-3`. If any of them skips or fails, `job-4` also skips.
- However, due to `if: always()`, `job-4` runs irrespective of the success of its dependencies i.e. even if any of its dependencies fails or skips, `job-4` would execute.
116 changes: 116 additions & 0 deletions .github/context-and-variables.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
## Idea
Everytime we execute a workflow, a GitHub Context object is created. It contains information about the workflow. For details, check [this](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/accessing-contextual-information-about-workflow-runs#about-contexts)

### Displaying the GitHub Context object
```yaml
name: Displaying GitHub Context Object
on: workflow_dispatch

jobs:
job-1:
runs-on: ubuntu-latest

steps:
- name: Print GitHub Context
env: # this env context stores variables only available to current step

# loc_var is the name of the variable. It is only accessible within this step
loc_var: ${{ toJSON(github) }} # toJSON is a built-in function that returns a pretty-print JSON representation
run: echo $loc_var

```
Check out the different built-in functions offered by GitHub, [here](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/evaluate-expressions-in-workflows-and-actions#functions)

### Passing information between jobs
This is how we pass string data between jobs:

```yaml
name: Passing Information Between Jobs
on: workflow_dispatch

jobs:

job1:
runs-on: ubuntu-latest
steps:
- id: step1 # this id would be used to access the output

run: echo "test=hello" >> $GITHUB_OUTPUT # $GITHUB_OUTPUT is a default env variable that stores the path on the runner to the file that sets the current step's outputs. It is unique for each step.

- id: step2

# basically, we are appending a variable declaration to the file pointed by $GITHUB_OUTPUT
run: echo "test=world" >> $GITHUB_OUTPUT


outputs: # mapping the outputs to variables

firstOp: ${{ steps.step1.outputs.test }} # firstOp stores hello
secondOp: ${{ steps.step2.outputs.test }} # secondOp stores world

job2:
runs-on: ubuntu-latest

needs: job1 # to access the outputs of a particular job, the current job must depend on it

steps:
- run: echo ${{needs.job1.outputs.firstOp}} ${{needs.job1.outputs.secondOp}}
```
### Variables
Variables provide a way to store and reuse non-sensitive configuration information. You can store any configuration data such as compiler flags, usernames, or server names as variables.
You can set your own custom variables or use the default environment variables that GitHub sets automatically. For more information, see [this](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/store-information-in-variables#default-environment-variables).
To set a custom environment variable for a single workflow, you can define it using the `env` key. The scope of a custom variable set by this method is limited to the element in which it is defined.
Variables can be scoped for:
- The entire workflow, by using `env` at the top level of the workflow file.
- The contents of a job within a workflow.
- A specific step within a job.

```yaml
name: Creating Variables
on: workflow_dispatch
env: # these variables would be created in all the runner instances, hence accessible throughout the entire workflow.
VAR1: myworkflowvar1
VAR2: myworkflowvar2
VAR3: myworkflowvar3
jobs:
job1:
runs-on: ubuntu-latest
env: # these variables are accessible only within this job
VAR2: myjobvar2
VAR3: myjobvar3
steps:
# we can also access a variable as $variable_name or ${{ env.variable_name }}
- run: |
echo value of VAR1 is $VAR1
echo value of VAR1 is ${{ env.VAR1 }}
echo value of VAR2 is $VAR2
echo value of VAR3 is $VAR3
env: # this variable is accessible only within this step
VAR3: mystepvar3
```
When accessing a variable, the lookup follows this order:

1. **Step-Level Variable**: Highest precedence; if defined, it overrides job- and workflow-level variables of the same name.
1. **Job-Level Variable**: Overrides workflow-level variables of the same name for all steps in the job.
1. **Workflow-Level Variable**: Lowest precedence; used if no job or step overrides are present.

*Variables defined at a lower level (step) are not available to higher levels (job or workflow).*

Hence, in the above example, the output is as follows:

```bash
```
2 changes: 1 addition & 1 deletion .github/workflows/basic-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ name: CI # name of the workflow
# Controls when the action will run
on: workflow_dispatch # to trigger the workflow manually (with a button), we use this event. This event will only trigger a workflow run if the workflow file exists on the default branch.

jobs: # This workflow contains a single job called "build"
jobs: # This workflow contains a single job having id "build"

build: # id of this job

Expand Down
30 changes: 30 additions & 0 deletions .github/workflows/passingInformation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Passing Information Between Jobs
on: workflow_dispatch

jobs:

job1:
runs-on: ubuntu-latest
steps:
- id: step1 # this id would be used to access the output

run: echo "test=hello" >> $GITHUB_OUTPUT # $GITHUB_OUTPUT is a default env variable that stores the path on the runner to the file that sets the current step's outputs. It is unique for each step.

- id: step2

# basically, we are appending a variable declaration to the file pointed by $GITHUB_OUTPUT
run: echo "test=world" >> $GITHUB_OUTPUT


outputs: # mapping the outputs to variables

firstOp: ${{ steps.step1.outputs.test }} # firstOp stores hello
secondOp: ${{ steps.step2.outputs.test }} # secondOp stores world

job2:
runs-on: ubuntu-latest

needs: job1 # to access the outputs of a particular job, the current job must depend on it

steps:
- run: echo ${{needs.job1.outputs.firstOp}} ${{needs.job1.outputs.secondOp}}
30 changes: 30 additions & 0 deletions .github/workflows/variables.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Creating Variables
on: workflow_dispatch

env: # these variables would be created in all the runner instances, hence accessible throughout the entire workflow.
VAR1: myworkflowvar1
VAR2: myworkflowvar2
VAR3: myworkflowvar3

jobs:

job1:
runs-on: ubuntu-latest

env: # these variables are accessible only within this job
VAR2: myjobvar2
VAR3: myjobvar3

steps:

# we can also access a variable as $variable_name or ${{ env.variable_name }}

- run: |
echo value of VAR1 is $VAR1
echo value of VAR1 is ${{ env.VAR1 }}
echo value of VAR2 is $VAR2
echo value of VAR3 is $VAR3
env: # this variable is accessible only within this step
VAR3: mystepvar3

0 comments on commit 6f3a8c7

Please sign in to comment.