Add support for packed modules to keep deployments <250mb #38
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| function private:Import-ModuleArchive { | ||
| <# | ||
| .SYNOPSIS | ||
| Unpacks compressed PowerShell modules from .zip archives (modules.zip) | ||
| .DESCRIPTION | ||
| Unpacks compressed PowerShell modules from .zip archives (modules.zip) into a subdirectory of /tmp. | ||
| This folder is later added to $env:PSModulePath, before user code runs, if module archives existed. | ||
| .NOTES | ||
| The contents of this archive should match the format of a folder in $Env:PSModulePath. More specifically: | ||
| * Module names should be top-level directories. | ||
| * One or more versions of the same module may be hosted in their own subdirectories, with respective version numbers. | ||
| * The module root (.psd1/.psm1 files, etc.) is contained within either the module-named or module-versioned directory. | ||
| Module packages are imported from two locations, from lowest to highest precedence: | ||
| * /opt/ (Combined Lambda layer directory) | ||
| * $Env:LAMBDA_TASK_ROOT (Lambda Function Package deployment directory) | ||
| If archives are detected at both locations, they will be extracted over the top of each-other. | ||
| #> | ||
|
|
||
| $SearchPaths = @( | ||
| "/opt/modules.zip" | ||
| $(Join-Path $env:LAMBDA_TASK_ROOT -ChildPath "modules.zip") | ||
| ) | ||
|
|
||
| If ($SearchPaths | ? { Test-Path $_ }) { | ||
sean-r-williams marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| $UnpackDirectory = '/tmp/powershell-custom-runtime-unpacked-modules/combined/' | ||
|
||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host '[RUNTIME-Import-ModuleArchive]Creating unpack directory for combined module archives' } | ||
| New-Item -ItemType Directory -Path $UnpackDirectory -Force | ||
sean-r-williams marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| $SearchPaths | ? { Test-Path $_ } | ForEach-Object { | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host "[RUNTIME-Import-ModuleArchive]Unpacking $_ to $UnpackDirectory" } | ||
| Expand-Archive -LiteralPath $_ -DestinationPath $UnpackDirectory -Force | ||
| } | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host '[RUNTIME-Import-ModuleArchive]Archive unpack complete' } | ||
| } | ||
| else { | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host '[RUNTIME-Import-ModuleArchive]No module archives detected; nothing to do.' } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| function private:Import-ModulePackage { | ||
| <# | ||
| .SYNOPSIS | ||
| Installs compressed PowerShell modules from NuGet packages (*.nupkg) | ||
| .DESCRIPTION | ||
| Installs compressed PowerShell modules from NuGet packages (*.nupkg) into a subdirectory of /tmp. | ||
| This folder is later added to $env:PSModulePath, before user code runs, if module packages existed. | ||
| .NOTES | ||
| These packages should match the NuPkg format used by PSResourceGet or PowerShellGet. | ||
| Packages can be exported either by: | ||
| * Downloading the .nupkg files directly from an upstream source (e.g. PowerShell Gallery) | ||
| * Using the -AsNuPkg parameter on Save-PSResource in the Microsoft.PowerShell.PSResourceGet module. | ||
| Module packages are imported from two locations, from lowest to highest precedence: | ||
| * /opt/module-nupkgs/ (Combined Lambda layer directory) | ||
| * $Env:LAMBDA_TASK_ROOT/module-nupkgs/ (Lambda Function Package deployment directory) | ||
| #> | ||
| $SearchPaths = @{ | ||
| Layer = "/opt/module-nupkgs/*.nupkg" | ||
| Root = (Join-Path $env:LAMBDA_TASK_ROOT -ChildPath "module-nupkgs" -AdditionalChildPath "*.nupkg") | ||
| } | ||
|
|
||
| If ($SearchPaths.Values | ? { Test-Path $_ }) { | ||
| $UnpackDirectory = '/tmp/powershell-custom-runtime-unpacked-modules/nupkgs/' | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host '[RUNTIME-Import-ModulePackage]Creating unpack directory for individual module packages' } | ||
| New-Item -ItemType Directory -Path $UnpackDirectory -Force | ||
| $SearchPaths.GetEnumerator() | ? { Test-Path $_.Value } | ForEach-Object { | ||
| $PackageDirectory = Split-Path $_.Value -Parent | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host "[RUNTIME-Import-ModulePackage]Importing module packages from $PackageDirectory" } | ||
| $RepositoryName = "Lambda-Local-$($_.Key)" | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host "[RUNTIME-Import-ModulePackage]Registering local package repository $RepositoryName" } | ||
| Register-PSResourceRepository -Name $RepositoryName -Uri $PackageDirectory -Trusted -Priority 1 | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host "[RUNTIME-Import-ModulePackage]Enumerating packages in $PackageDirectory (PSResource repository $RepositoryName)" } | ||
| Find-PSResource -Name * -Repository $RepositoryName | ForEach-Object -Parallel { | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host "[RUNTIME-Import-ModulePackage]Saving package $($_.Name) version $($_.Version) (PSResource repository $($using:RepositoryName))" } | ||
| $_ | Save-PSResource -SkipDependencyCheck -Path $using:PackageDirectory -Quiet -AcceptLicense -Confirm:$false | ||
| } | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host "[RUNTIME-Import-ModulePackage]Registering local package repository $RepositoryName" } | ||
| Unregister-PSResourceRepository -Name $RepositoryName -Confirm:$false | ||
| } | ||
|
||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host '[RUNTIME-Import-ModulePackage]Archive unpack complete' } | ||
| } | ||
| else { | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host '[RUNTIME-Import-ModulePackage]No module archives detected; nothing to do.' } | ||
| } | ||
|
|
||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| function private:Test-RuntimePackedModule { | ||
| <# | ||
| .SYNOPSIS | ||
| Tests whether the current runtime environment contains compressed module packages (combined .zip or per-module .nupkg) | ||
| .DESCRIPTION | ||
| Tests whether the current runtime environment contains compressed module packages (combined .zip or per-module .nupkg) | ||
| .NOTES | ||
| Looks for module packages in two locations: | ||
| * /opt/ (Combined Lambda layer directory) | ||
| * $Env:LAMBDA_TASK_ROOT (Lambda Function package directory) | ||
| Module packages can take two forms: | ||
| * A single, combined module archive, named "modules.zip". | ||
| The contents of this archive should match the format of a folder in $Env:PSModulePath. | ||
| (Module names as top-level directories, optional version subdirectory, corresponding module root) | ||
| * Individual module archives, as .nupkg files, inside a subdirectory named "module-nupkgs" | ||
| These files should match: | ||
| * The naming convention used by PSResourceGet. (e.g. <module-name>.<version>.nupkg) | ||
| * The Nupkg archive spec (module root at archive root, NuGet [Content_Types].xml/_rels, etc.) | ||
| The following file locations should all be detected (assume $Env:LAMBDA_TASK_ROOT = /var/lambda/) | ||
| * /opt/modules.zip | ||
| * /var/lambda/modules.zip | ||
| * /opt/module-nupkgs/AWS.Tools.Common.4.1.833.nupkg | ||
| * /var/lambda/module-nupkgs/AWS.Tools.Common.4.1.833.nupkg | ||
| .EXAMPLE | ||
| Test-MyTestFunction -Verbose | ||
| Explanation of the function or its result. You can include multiple examples with additional .EXAMPLE lines | ||
| #> | ||
|
|
||
| [CmdletBinding()] | ||
| param( | ||
| # Looks for combined module archives (modules.zip). | ||
| [Parameter( | ||
| Mandatory, | ||
| ParameterSetName="Combined" | ||
| )] | ||
| [Switch] | ||
| $Combined, | ||
|
|
||
| # Looks for individual module packages (*.nupkg). | ||
| [Parameter( | ||
| Mandatory, | ||
| ParameterSetName="NuPkg" | ||
| )] | ||
| [Switch] | ||
| $NuPkg | ||
| ) | ||
|
|
||
| $BaseDirectories = @( | ||
| "/opt", | ||
| $Env:LAMBDA_TASK_ROOT | ||
| ) | ||
|
||
|
|
||
| switch ($PSCmdlet.ParameterSetName) { | ||
| "Combined" { | ||
|
|
||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host "[RUNTIME-Test-RuntimePackedModule]Searching for combined module archives" } | ||
|
|
||
| $BaseDirectories | Join-Path -ChildPath "modules.zip" | Get-Item -ErrorAction SilentlyContinue | Set-Variable FoundItems | ||
|
|
||
| } | ||
| "NuPkg" { | ||
|
|
||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host "[RUNTIME-Test-RuntimePackedModule]Searching for individual module packages" } | ||
|
|
||
| $BaseDirectories | Join-Path -ChildPath "module-nupkgs" -AdditionalChildPath "*.nupkg" | Get-Item -ErrorAction SilentlyContinue | Set-Variable FoundItems | ||
|
|
||
| } | ||
| } | ||
|
|
||
| If ($FoundItems) { | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host "[RUNTIME-Test-RuntimePackedModule]Found $($FoundItems | Measure-Object | % Count) match(es)" } | ||
| return $true | ||
| } else { | ||
| if ($env:POWERSHELL_RUNTIME_VERBOSE -eq 'TRUE') { Write-Host "[RUNTIME-Test-RuntimePackedModule]No matches found" } | ||
| return $false | ||
| } | ||
| } | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For performance reasons (reading from disk is relatively slow), it would be great to merge these into a single call, that looks for both
.zipand.nupkgfiles, and passes the files found into appropriate import functions.As it stands, this will traverse the file system twice (in each of the
Test-RuntimePackedModulecalls, and then if something is found, again in the relativeImport-*functions).Perhaps the
Test-RuntimePackedModulecould return the specific paths to the files found, and theImport-*functions could accept a string of path values as a parameters reducing the disk reads.