Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 25 additions & 23 deletions README-DEV.rst
Original file line number Diff line number Diff line change
Expand Up @@ -278,31 +278,32 @@ suites, and let Clouseau automatically manage them.
Although the ``./configure`` and the ``./dev/run`` scripts try to take
care of the details of the Clouseau deployment, it is still the
responsibility of the user to provide a suitable Java environment for
running. Clouseau can run with JRE 1.7 and 1.8 only. Also, when
Nouveau is in use, which uses a more recent Java environment, the old
JDK has to be installed separately and the ``CLOUSEAU_JAVA_HOME``
environment variable has to be set to point its location.
running. Depending on the version, Clouseau can run with either JRE
1.7 and 1.8 only, when 2.x is selected, or JRE 21 for 3.x. Also, when
Nouveau is installed, which might require a different Java
environment, the one specific for Clouseau has to be installed
separately and the ``CLOUSEAU_JAVA_HOME`` environment variable has to
be set to point its location.

Fortunately, the ```asdf`` tool <https://asdf-vm.com/>` provides a
convenient way to install old versions of JDK through its ```java``
plugin <https://github.com/halcyon/asdf-java>`::
convenient way to install different versions of JDK through its
```java`` plugin <https://github.com/halcyon/asdf-java>`::

asdf plugin add java

Then use ``asdf`` to install it::
Then use ``asdf`` to install it, for example::

asdf install java zulu-jre-8.74.0.17
asdf install java openjdk-21

Finally, use ``asdf`` to set the ``CLOUSEAU_JAVA_HOME`` environment
variable::

export CLOUSEAU_JAVA_HOME=$(asdf where java zulu-jre-8.74.0.17)
export CLOUSEAU_JAVA_HOME=$(asdf where java openjdk-21)

If the use of ``asdf`` is not an option, `the Zulu site
<https://cdn.azul.com/zulu/bin/>` could be used directly to get the
distribution package for the appropriate JRE version. But this is
just one of the possibilities to access installers for old Java
environments.
If the use of ``asdf`` is not an option, any standard Java installer
would do it, and the Zulu site <https://cdn.azul.com/zulu/bin/>` could
be used directly to get the distribution package for the older JRE and
JDK versions, for example.

Once both Clouseau and the corresponding Java environment are set,
they are not put in use automatically. In order to do so, the
Expand Down Expand Up @@ -333,15 +334,16 @@ respective test targets, e.g. ``mango-test``::
This can even be done if there was no local Clouseau deployment
configured previously. Mind that this will require building Clouseau
from source, which causes the nodes start up somewhat slower. It also
requires JDK 1.7 and Apache Maven 3.8 to be present, that is why it is
important the set the ``CLOUSEAU_JAVA_HOME`` and the
``CLOUSEAU_M2_HOME`` environment variables accordingly, for instance::

asdf install java zulu-7.56.0.11
asdf plugin add maven
asdf install maven 3.8.8
export CLOUSEAU_JAVA_HOME=$(asdf where java zulu-7.56.0.11)
export CLOUSEAU_M2_HOME=$(asdf where maven 3.8.8)
requires JDK 11 (or later) and SBT 1.10 (or later) to be present.
That is why it is important the set the ``CLOUSEAU_JAVA_HOME`` and the
``CLOUSEAU_SBT_HOME`` environment variables accordingly, for
instance::

asdf install java openjdk-21
asdf plugin add sbt
asdf install sbt 1.11.7
export CLOUSEAU_JAVA_HOME=$(asdf where java openjdk-21)
export CLOUSEAU_SBT_HOME=$(asdf where sbt 1.11.7)

Static Code Analysis
~~~~~~~~~~~~~~~~~~~~
Expand Down
38 changes: 31 additions & 7 deletions build-aux/Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ meta = [
spidermonkey_vsn: '60',
with_nouveau: true,
with_clouseau: true,
clouseau_java_home: '/usr',
quickjs_test262: true,
image: "apache/couchdbci-centos:8-erlang-${ERLANG_VERSION}"
],
Expand All @@ -49,6 +50,7 @@ meta = [
spidermonkey_vsn: '78',
with_nouveau: true,
with_clouseau: true,
clouseau_java_home: '/usr',
quickjs_test262: true,
image: "apache/couchdbci-centos:9-erlang-${ERLANG_VERSION}"
],
Expand All @@ -58,6 +60,7 @@ meta = [
spidermonkey_vsn: '91',
with_nouveau: true,
with_clouseau: true,
clouseau_java_home: '/opt/java/openjdk',
quickjs_test262: true,
image: "apache/couchdbci-ubuntu:jammy-erlang-${ERLANG_VERSION}"
],
Expand All @@ -67,6 +70,7 @@ meta = [
spidermonkey_vsn: '115',
with_nouveau: true,
with_clouseau: true,
clouseau_java_home: '/opt/java/openjdk',
quickjs_test262: true,
image: "apache/couchdbci-ubuntu:noble-erlang-${ERLANG_VERSION}"
],
Expand All @@ -76,6 +80,7 @@ meta = [
spidermonkey_vsn: '78',
with_nouveau: true,
with_clouseau: true,
clouseau_java_home: '/opt/java/openjdk',
quickjs_test262: true,
image: "apache/couchdbci-debian:bullseye-erlang-${ERLANG_VERSION}"
],
Expand Down Expand Up @@ -118,6 +123,7 @@ meta = [
spidermonkey_vsn: '78',
with_nouveau: true,
with_clouseau: true,
clouseau_java_home: '/opt/java/openjdk',
// Test this in in the bookworm-quickjs variant
quickjs_test262: false,
image: "${DOCKER_IMAGE_BASE}-${ERLANG_VERSION}"
Expand All @@ -128,6 +134,7 @@ meta = [
spidermonkey_vsn: '78',
with_nouveau: true,
with_clouseau: true,
clouseau_java_home: '/opt/java/openjdk',
quickjs_test262: false,
image: "${DOCKER_IMAGE_BASE}-${MAXIMUM_ERLANG_VERSION}"
],
Expand All @@ -137,6 +144,7 @@ meta = [
disable_spidermonkey: true,
with_nouveau: true,
with_clouseau: true,
clouseau_java_home: '/opt/java/openjdk',
quickjs_test262: true,
image: "${DOCKER_IMAGE_BASE}-${ERLANG_VERSION}"
],
Expand All @@ -154,6 +162,7 @@ meta = [
spidermonkey_vsn: '78',
with_nouveau: true,
with_clouseau: true,
clouseau_java_home: '/opt/java/openjdk',
// Test this in in the bookworm-quickjs variant
quickjs_test262: false,
image: "${DOCKER_IMAGE_BASE}-${ERLANG_VERSION}",
Expand All @@ -165,6 +174,7 @@ meta = [
spidermonkey_vsn: '128',
with_nouveau: true,
with_clouseau: true,
clouseau_java_home: '/opt/java/openjdk',
quickjs_test262: true,
image: "apache/couchdbci-debian:trixie-erlang-${ERLANG_VERSION}"
],
Expand All @@ -173,7 +183,7 @@ meta = [
name: 'FreeBSD x86_64',
spidermonkey_vsn: '91',
with_clouseau: true,
clouseau_java_home: '/usr/local/openjdk8-jre',
clouseau_java_home: '/usr/local/openjdk21',
quickjs_test262: false,
gnu_make: 'gmake'
],
Expand All @@ -184,7 +194,7 @@ meta = [
name: 'FreeBSD ARM64 QuickJS',
disable_spidermonkey: true,
with_clouseau: true,
clouseau_java_home: '/usr/local/openjdk8-jre',
clouseau_java_home: '/usr/local/openjdk21',
quickjs_test262: false,
gnu_make: 'gmake'
],
Expand All @@ -205,6 +215,7 @@ meta = [
name: 'Windows 2022',
spidermonkey_vsn: '128',
with_clouseau: true,
clouseau_java_home: /C:\tools\zulu21.46.19-ca-jdk21.0.9-win_x64/,
quickjs_test262: false,
node_label: 'win'
]
Expand Down Expand Up @@ -265,8 +276,8 @@ def generateNativeStage(platform) {
retry (3) {sh '$MAKE eunit'}
if (meta[platform].quickjs_test262) {retry(3) {sh 'make quickjs-test262'}}
retry (3) {sh '$MAKE elixir'}
retry (3) {sh '$MAKE elixir-search'}
retry (3) {sh '$MAKE mango-test'}
retry (3) {sh '$MAKE elixir-search ERLANG_COOKIE=crumbles'}
retry (3) {sh '$MAKE mango-test ERLANG_COOKIE=crumbles'}
retry (3) {sh '$MAKE weatherreport-test'}
retry (3) {sh '$MAKE nouveau-test'}
}
Expand Down Expand Up @@ -302,7 +313,12 @@ def generateNativeStage(platform) {
powershell( script: "New-Item -ItemType Directory -Path '${platform}/build' -Force", label: 'Create build directories' )
powershell( script: "tar -xf (Get-Item apache-couchdb-*.tar.gz) -C '${platform}/build' --strip-components=1", label: 'Unpack release' )
dir( "${platform}/build" ) {
withClouseau = meta[platform].with_clouseau ? '-WithClouseau' : ''
def withClouseau = meta[platform].with_clouseau ? '-WithClouseau' : ''
def setClouseauJavaHome = ''

if (meta[platform].clouseau_java_home) {
setClouseauJavaHome = "Set-Item -Path env:CLOUSEAU_JAVA_HOME -Value '${meta[platform].clouseau_java_home}'"
}

powershell( script: """
.\\..\\..\\couchdb-glazier\\bin\\shell.ps1
Expand All @@ -317,12 +333,14 @@ def generateNativeStage(platform) {
powershell( script: """
.\\..\\..\\couchdb-glazier\\bin\\shell.ps1
Set-Item -Path env:GRADLE_OPTS -Value '-Dorg.gradle.daemon=false'
${setClouseauJavaHome}
make -f Makefile.win elixir-search ERLANG_COOKIE=crumbles
""", label: 'Clouseau tests')

powershell( script: """
.\\..\\..\\couchdb-glazier\\bin\\shell.ps1
Set-Item -Path env:GRADLE_OPTS -Value '-Dorg.gradle.daemon=false'
${setClouseauJavaHome}
make -f Makefile.win mango-test ERLANG_COOKIE=crumbles
""", label: 'Mango tests')

Expand Down Expand Up @@ -382,14 +400,20 @@ def generateContainerStage(platform) {
sh( script: "mkdir -p ${platform}/build", label: 'Create build directories' )
sh( script: "tar -xf apache-couchdb-*.tar.gz -C ${platform}/build --strip-components=1", label: 'Unpack release' )
quickjs_tests262 = meta[platform].quickjs_test262
def setClouseauJavaHome = ''

if (meta[platform].clouseau_java_home) {
setClouseauJavaHome = "env CLOUSEAU_JAVA_HOME='${meta[platform].clouseau_java_home}' "
}

dir( "${platform}/build" ) {
sh "${configure(meta[platform])}"
sh 'make'
retry(3) {sh 'make eunit'}
if (meta[platform].quickjs_test262) {retry(3) {sh 'make quickjs-test262'}}
retry(3) {sh 'make elixir'}
retry(3) {sh 'make elixir-search'}
retry(3) {sh 'make mango-test'}
retry(3) {sh "${setClouseauJavaHome}timeout 5m make elixir-search ERLANG_COOKIE=crumbles"}
retry(3) {sh "${setClouseauJavaHome}timeout 5m make mango-test ERLANG_COOKIE=crumbles"}
retry(3) {sh 'make weatherreport-test'}
retry(3) {sh 'make nouveau-test'}
}
Expand Down
36 changes: 25 additions & 11 deletions configure
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ WITH_FAUXTON="true"
WITH_DOCS="true"
WITH_NOUVEAU="false"
WITH_CLOUSEAU="false"
WITH_ZIOSE="true"
ERLANG_MD5="false"
SKIP_DEPS="false"
WITH_SPIDERMONKEY="true"
Expand All @@ -43,7 +44,7 @@ JS_ENGINE=${JS_ENGINE:-"spidermonkey"}
SM_VSN=${SM_VSN:-"91"}
CLOUSEAU_MTH=${CLOUSEAU_MTH:-"dist"}
CLOUSEAU_URI=${CLOUSEAU_URI:-"https://github.com/cloudant-labs/clouseau/releases/download/%s/clouseau-%s-dist.zip"}
CLOUSEAU_VSN=${CLOUSEAU_VSN:-"2.25.0"}
CLOUSEAU_VSN=${CLOUSEAU_VSN:-"3.0.0"}
CLOUSEAU_DIR="$(pwd)"/clouseau
ARCH="$(uname -m)"
MULTIARCH_NAME="$(command -v dpkg-architecture > /dev/null && dpkg-architecture -q DEB_HOST_MULTIARCH || true)"
Expand Down Expand Up @@ -314,6 +315,10 @@ parse_opts() {

parse_opts $@

if [ "$(echo "$CLOUSEAU_VSN" | cut -d '.' -f 1)" -lt "3" ]; then
WITH_ZIOSE="false"
fi

if [ "${WITH_SPIDERMONKEY}" = "true" ] && [ "${ARCH}" = "aarch64" ] && [ "${SM_VSN}" = "60" ]; then
echo "ERROR: SpiderMonkey 60 is known broken on ARM 64 (aarch64). Use another version instead."
exit 1
Expand Down Expand Up @@ -488,14 +493,17 @@ install_local_clouseau() {
case "$CLOUSEAU_MTH" in
dist)
_DIST_URL=$(printf "$CLOUSEAU_URI" "$CLOUSEAU_VSN" "$CLOUSEAU_VSN")
_MAVEN_BASE_URI=https://repo1.maven.org/maven2
_LOGBACK_DIST_URL="$_MAVEN_BASE_URI"/ch/qos/logback

: "${LOGBACK_VERSION:=1.2.13}"
_LOGBACK_CORE_JAR=logback-core-"$LOGBACK_VERSION".jar
_LOGBACK_CORE_URL="$_LOGBACK_DIST_URL"/logback-core/"$LOGBACK_VERSION"/"$_LOGBACK_CORE_JAR"
_LOGBACK_CLASSIC_JAR=logback-classic-"$LOGBACK_VERSION".jar
_LOGBACK_CLASSIC_URL="$_LOGBACK_DIST_URL"/logback-classic/"$LOGBACK_VERSION"/"$_LOGBACK_CLASSIC_JAR"
if [ "${WITH_ZIOSE}" = "false" ]; then
_MAVEN_BASE_URI=https://repo1.maven.org/maven2
_LOGBACK_DIST_URL="$_MAVEN_BASE_URI"/ch/qos/logback

: "${LOGBACK_VERSION:=1.2.13}"
_LOGBACK_CORE_JAR=logback-core-"$LOGBACK_VERSION".jar
_LOGBACK_CORE_URL="$_LOGBACK_DIST_URL"/logback-core/"$LOGBACK_VERSION"/"$_LOGBACK_CORE_JAR"
_LOGBACK_CLASSIC_JAR=logback-classic-"$LOGBACK_VERSION".jar
_LOGBACK_CLASSIC_URL="$_LOGBACK_DIST_URL"/logback-classic/"$LOGBACK_VERSION"/"$_LOGBACK_CLASSIC_JAR"
fi

check_local_clouseau_dir "$CLOUSEAU_DIR"
mkdir -p "$CLOUSEAU_DIR"
Expand All @@ -511,10 +519,16 @@ install_local_clouseau() {

rm clouseau.zip

echo "Fetching Logback $LOGBACK_VERSION from $_LOGBACK_DIST_URL..."
if [ "${WITH_ZIOSE}" = "false" ]; then
echo "Fetching Logback $LOGBACK_VERSION from $_LOGBACK_DIST_URL..."

fetch_file "$CLOUSEAU_DIR"/"$_LOGBACK_CORE_JAR" "$_LOGBACK_CORE_URL"
fetch_file "$CLOUSEAU_DIR"/"$_LOGBACK_CLASSIC_JAR" "$_LOGBACK_CLASSIC_URL"
fetch_file "$CLOUSEAU_DIR"/"$_LOGBACK_CORE_JAR" "$_LOGBACK_CORE_URL"
fetch_file "$CLOUSEAU_DIR"/"$_LOGBACK_CLASSIC_JAR" "$_LOGBACK_CLASSIC_URL"
fi

if [ "${WITH_ZIOSE}" = "true" ]; then
touch "$CLOUSEAU_DIR"/.ziose
fi
;;

git)
Expand Down
38 changes: 24 additions & 14 deletions configure.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ Param(
[ValidateNotNullOrEmpty()]
[string]$ClouseauMethod = "dist", # method for Clouseau to deploy: git or dist (default dist)
[ValidateNotNullOrEmpty()]
[string]$ClouseauVersion = "2.25.0", # select the version of Clouseau to use (default 2.25.0)
[string]$ClouseauVersion = "3.0.0", # select the version of Clouseau to use (default 3.0.0)
[ValidateNotNullOrEmpty()]
[string]$ClouseauUri = "https://github.com/cloudant-labs/clouseau/releases/download/{0}/clouseau-{0}-dist.zip", # location for retrieving Clouseau (default https://github.com/cloudant-labs/clouseau/releases/download/2.25.0/clouseau-2.25.0-dist.zip)
[ValidateNotNullOrEmpty()]
Expand Down Expand Up @@ -150,6 +150,7 @@ $BuildDocs = (-not $DisableDocs).ToString().ToLower()
$WithProper = (-not $DisableProper).ToString().ToLower()
$ErlangMD5 = ($EnableErlangMD5).ToString().ToLower()
$WithSpiderMonkey = (-not $DisableSpiderMonkey).ToString().ToLower()
$WithZIOSE = [int]($ClouseauVersion.Split('.')[0]) -ge 3

if ($JSEngine -eq "quickjs") {
$WithSpiderMonkey = "false"
Expand Down Expand Up @@ -333,12 +334,15 @@ if ($WithClouseau)

New-Item -Path $ClouseauDir -ItemType Directory | Out-Null

$LogbackVersion = "1.2.13"
$ClouseauDistUrl = $ClouseauUri -f $ClouseauVersion
$LogbackCoreJar = "logback-core-$LogbackVersion.jar"
$LogbackCoreJarUrl = "https://repo1.maven.org/maven2/ch/qos/logback/logback-core/$LogbackVersion/$LogbackCoreJar"
$LogbackClassicJar = "logback-classic-$LogbackVersion.jar"
$LogbackClassicJarUrl = "https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/$LogbackVersion/$LogbackClassicJar"

If (-not $WithZIOSE) {
$LogbackVersion = "1.2.13"
$LogbackCoreJar = "logback-core-$LogbackVersion.jar"
$LogbackCoreJarUrl = "https://repo1.maven.org/maven2/ch/qos/logback/logback-core/$LogbackVersion/$LogbackCoreJar"
$LogbackClassicJar = "logback-classic-$LogbackVersion.jar"
$LogbackClassicJarUrl = "https://repo1.maven.org/maven2/ch/qos/logback/logback-classic/$LogbackVersion/$LogbackClassicJar"
}

Set-Variable ProgressPreference SilentlyContinue
Invoke-WebRequest -MaximumRedirection 1 -OutFile clouseau.zip $ClouseauDistUrl
Expand All @@ -356,16 +360,22 @@ if ($WithClouseau)
Remove-Item "$ClouseauDir\clouseau-$ClouseauVersion"
Remove-Item clouseau.zip

Invoke-WebRequest -MaximumRedirection 1 -OutFile "$ClouseauDir\$LogbackCoreJar" $LogbackCoreJarUrl
If ($LASTEXITCODE -ne 0) {
Write-Output "ERROR: $LogbackCoreJarUrl could not be downloaded."
exit 1
If (-not $WithZIOSE) {
Invoke-WebRequest -MaximumRedirection 1 -OutFile "$ClouseauDir\$LogbackCoreJar" $LogbackCoreJarUrl
If ($LASTEXITCODE -ne 0) {
Write-Output "ERROR: $LogbackCoreJarUrl could not be downloaded."
exit 1
}

Invoke-WebRequest -MaximumRedirection 1 -OutFile "$ClouseauDir\$LogbackClassicJar" $LogbackClassicJarUrl
If ($LASTEXITCODE -ne 0) {
Write-Output "ERROR: $LogbackClassicJarUrl could not be downloaded."
exit 1
}
}

Invoke-WebRequest -MaximumRedirection 1 -OutFile "$ClouseauDir\$LogbackClassicJar" $LogbackClassicJarUrl
If ($LASTEXITCODE -ne 0) {
Write-Output "ERROR: $LogbackClassicJarUrl could not be downloaded."
exit 1
If ($WithZIOSE) {
New-Item -ItemType File -Path "$ClouseauDir\.ziose"
}
}
elseif ($ClouseauMethod -eq "git") {
Expand Down
Loading