doc updates

mitchcapper · Dec 18, 2024 · d6296b3 · d6296b3
1 parent 77013a3
commit d6296b3
Show file tree

Hide file tree

Showing 3 changed files with 78 additions and 7 deletions.
diff --git a/DEVELOPERS.md b/DEVELOPERS.md
@@ -14,6 +14,8 @@ The helpers are where the bulk of code is consolidated down that we would use fo
 - helpers_vcpkg.sh - handles installing and removing vcpkg packages, adding them to the proper path/env/includes even installs vcpkg is not installed (we use a local copy separate from the system to avoid polluting the users vcpkg.
 - helpers_ini.sh - INI reader and config merge/templating tool.  It also handles exporting the final config to a file so it can be consumed by other things (ie our visual studio sln generator).
 
+# Custom build steps / blocks
+In an effort to create cleaner build scripts some code was moved more behind the scenes in non obvious ways.  For example many steps will start the block with: `if [[ -z $SKIP_STEP || $SKIP_STEP == "STEPNAME" ]]; then` this means if no step is specified or if it is the step to resume at go into the block.  Easy enough.  What needs to happen inside that block is `SKIP_STEP=""` otherwise it won't proceed to the next step properly on resumes.  The built in steps this happens automatically on, generally the first function they call has a `SKIP_STEP=""` line.   The other item, purely for error messages, is "CUR_STEP".  If yo want to set this on entering a step to the step name and then set it back to empty at the end if there is an error it will say "you may be able to resume by passing step X to the build script.  This is not required though, purely a user help thing.
 
 # debugging build failures
 
@@ -30,4 +32,12 @@ It is not recommended to run most commands manually but rather try to use the re
 # patch generation
 Easy way to generate the patch from modified repo, go to your modified branch (make sure code committed) and run: git diff --color=never master > repo_NAME.patch
 
-You will want to preserve whitespace when generating the patch but we will ignore whne applying.
+You will want to preserve whitespace when generating the patch but we will ignore whne applying.
+
+
+# GitHub Actions
+The github action build scripts use some cheats to work. For one, because we can't wait cross workflow completions deps are done based on the last flow.  If openssl depends on our zstd then the zstd copy it uses to build is the one from the github action prior to the current one.  This past artifact use means if a bug is fixed in a dep it may require two commits to show in the final product. Technically this can have a recursive effect.  If A depends on B which depends on C and a bug is fixed in C it may take 3 commits to reach A (first commit fixes it in C and generates the C artifact, next commit results in B being built with the updated artifact, and finally the 3rd commit results in C picking it up).  This only applies when one dep completely includes the other.  If B just references A and C also references A then in two commits C will pick it up.
+
+In theory we could build on non-windows platforms but right now we dont try to do so as I am not sure there is much reason.  There would also need to be some windows specific changes to the helpers gated properly.
+
+Right now we build static versions in debug and release configurations for every library.
diff --git a/build/README.md b/build/README.md
@@ -162,7 +162,7 @@ Add autoconf section before configure:
 	if [[ -z $SKIP_STEP || $SKIP_STEP == "autoconf" ]]; then #not empty allowed as if we bootstrapped above we dont need to run nautoconf
 		gnulib_ensure_buildaux_scripts_copied
 		autoreconf --symlink --verbose --install
-		libtool_fixes "build-aux/ltmain.sh" "m4/libtool.m4"
+		libtool_fixes
 		autoreconf --verbose #update for libtool fixes
 		SKIP_STEP=""
 	fi
@@ -207,14 +207,14 @@ Right before configure:
 
 ## Gawk
 ### Template Script Args
-`--BUILD_NAME gawk --GitRepo https://git.savannah.gnu.org/git/gawk.git --GNU_LIBS_ADD_TO_REPO 1 --BUILD_MSVC_RUNTIME_INFO_ADD_TO_C_AND_LDFLAGS=1 --PREFER_STATIC_LINKING=0 --GNU_LIBS_ADDL "mkstemp" "fts" "sys_socket" "strcasestr" "regex" "random" "flexmember" "setlocale" "locale" "dfa" "sleep" "strsignal" "sys_ioctl" "connect" "listen" "accept" "fnmatch-h" "fnmatch-gnu" "recvfrom" "bind" "setsockopt" "getsockopt" "getopt-gnu" "shutdown" "sys_random" "popen" "pclose" "socket" "strcase" "timegm" "setenv" "unsetenv" "usleep" "fprintf-gnu" --BUILD_MSVC_IGNORE_WARNINGS 4068 --CONFIG_CMD_ADDL "ac_cv_search_dlopen=`"none required`"" "--enable-extensions" "--enable-threads=windows" "acl_shlibext=dll" "ac_cv_header_dlfcn_h=yes"  --BUILD_MAKE_CMD_ADDL 'DEFPATH="\"./;%%PROGRAMDATA%%/gawk/share\""' 'DEFLIBPATH="\"./;%%PROGRAMDATA%%/gawk/lib\""'`
+`--BUILD_NAME gawk --GitRepo https://git.savannah.gnu.org/git/gawk.git --GNU_LIBS_ADD_TO_REPO 1 --BUILD_MSVC_RUNTIME_INFO_ADD_TO_C_AND_LDFLAGS=1 --PREFER_STATIC_LINKING=0 --GNU_LIBS_ADDL "mkstemp" "fts" "sys_socket" "strcasestr" "regex" "random" "flexmember" "setlocale" "locale" "dfa" "sleep" "strsignal" "sys_ioctl" "connect" "listen" "accept" "fnmatch-h" "fnmatch-gnu" "recvfrom" "bind" "setsockopt" "getsockopt" "getopt-gnu" "shutdown" "sys_random" "popen" "pclose" "socket" "strcase" "timegm" "setenv" "unsetenv" "usleep" "fprintf-gnu" --BUILD_MSVC_IGNORE_WARNINGS 4068 --CONFIG_CMD_ADDL "ac_cv_search_dlopen=-luser32" "--enable-extensions" "--enable-threads=windows" "acl_shlibext=dll" "ac_cv_header_dlfcn_h=yes"  --BUILD_MAKE_CMD_ADDL 'DEFPATH="\"./;%%PROGRAMDATA%%/gawk/share\""' 'DEFLIBPATH="\"./;%%PROGRAMDATA%%/gawk/lib\""'`
 
 ### Modifications
-We specifically set it to not statically link.  It will compile with static linking fine, but extensions are also compiled statically into libs which obviously gawk can't use at runtime then.  The only dependency gawk has when dynamically compiled is the vc runtime.
+We specifically set it to not statically link.  It will compile with static linking fine, but extensions are also compiled statically into libs which obviously gawk can't use at runtime then.  The only dependency gawk has when dynamically compiled is the vc runtime.  As for the ac_cv_search_dlopen setting it to something will bypass the check we would fail, but the none-required value no longer works so we just set it to a lib we will include anyway.
 
 In the clone step add
 ```bash
-cp gnulib/build-aux/bootstrap .
+		cp gnulib/build-aux/bootstrap .
 		cp gnulib/build-aux/bootstrap.conf .
 		echo "gnulib_tool_option_extras=\" --without-tests --symlink --m4-base=m4 --lib=libgawk --source-base=lib --cache-modules\"" >> bootstrap.conf
 		git mv m4 m4_orig
@@ -499,10 +499,11 @@ make ships with a bunch of gnulib baked in but doesn't use gnulib proper we chan
 
 ## gnutls
 ### Template Script Args
-`--BUILD_NAME gnutls --GitRepo https://github.com/gnutls/gnutls.git --GNU_LIBS_ADDL "dirent" "getopt-gnu" --CONFIG_CMD_ADDL "--with-included-unistring" "--with-included-libtasn1" "--without-p11-kit" --VC_PKGDEPS "gmp" "nettle" "brotli" "zstd"  --PKG_CONFIG_MANUAL_ADD "gmp" --BUILD_ADDL_CFLAGS "-I../gl/"`
+`--BUILD_NAME gnutls --GitRepo https://github.com/gnutls/gnutls.git --GNU_LIBS_ADDL "dirent" "getopt-gnu" --CONFIG_CMD_ADDL "--with-included-unistring"  "--without-p11-kit" --VC_PKGDEPS "gmp" "nettle" "brotli" "zstd"  --PKG_CONFIG_MANUAL_ADD "gmp" --BUILD_ADDL_CFLAGS "-I../gl/" --OUR_LIB_DEPS "libtasn1" --OUR_LIB_BINS_PATH "libtasn1"`
 
 ### Modifications
 At top after startcommon add: `BLD_CONFIG_GNU_LIBS_EXCLUDE=("${BLD_CONFIG_GNU_LIBS_DEFAULT[@]}")` have to wait until then as otherwise full template sub not done
+Would prefer if --with-included-libtasn1 would work but for some reason it doesn't seem to still use it so we compile it ourselves instead.
 
 After switch_to_master_and_patch add:
 ```bash
@@ -515,7 +516,7 @@ After switch_to_master_and_patch add:
 				sed -i -E '/doc\//d;/GTK_DOC_CHECK/d' configure.ac
 			fi
 			if [[ $BLD_CONFIG_CONFIG_NO_PO -eq 1 ]]; then
-			sed -i -E '/SUBDIRS \+= po/d' Makefile.am
+				sed -i -E '/SUBDIRS \+= po/d' Makefile.am
 				sed -i -E '/po\//d;' configure.ac
 			fi
 			if [[ -f "gl/override/doc/gendocs_template.diff" && $BLD_CONFIG_CONFIG_NO_DOCS -eq "1" ]]; then
@@ -525,3 +526,62 @@ After switch_to_master_and_patch add:
 
 Before configure:
 `ensure_perl_installed_set_exports "AS"`
+
+## libtasn1
+### Template Script Args
+`--BUILD_NAME libtasn1 --GitRepo https://gitlab.com/gnutls/libtasn1.git --HaveOurPatch=0 --BUILD_ADDL_CFLAGS_STATIC -DASN1_STATIC`
+
+### modifications
+At top after startcommon add: `BLD_CONFIG_GNU_LIBS_EXCLUDE=("${BLD_CONFIG_GNU_LIBS_DEFAULT[@]}")` have to wait until then as otherwise full template sub not done
+
+After switch_to_master_and_patch add:
+```bash
+			if [[ $BLD_CONFIG_CONFIG_NO_TESTS -eq 1 ]]; then
+				sed -i -E '/SUBDIRS \+= tests/d' Makefile.am
+				sed -i -E '/tests\//d;/fuzz\//d;' configure.ac
+			fi
+			if [[ $BLD_CONFIG_CONFIG_NO_DOCS -eq 1 ]]; then
+				sed -i -E '/enable-doc/d;/enable-gtk-doc/d;/SUBDIRS \+= doc/d;' Makefile.am
+				sed -i -E '/doc\//d;/GTK_DOC_CHECK/d' configure.ac
+			fi
+			if [[ $BLD_CONFIG_CONFIG_NO_PO -eq 1 ]]; then
+				sed -i -E '/SUBDIRS \+= po/d' Makefile.am
+				sed -i -E '/po\//d;' configure.ac
+			fi
+			if [[ -f "gl/override/doc/gendocs_template.diff" && $BLD_CONFIG_CONFIG_NO_DOCS -eq "1" ]]; then
+				git rm "gl/override/doc/gendocs_template.diff"
+			fi
+```
+
+## p11-kit
+
+### Template Script Args
+`--BUILD_NAME p11-kit --GitRepo https://github.com/p11-glue/p11-kit --HaveOurPatch=0  --GNU_LIBS_USED=0`
+
+### modifications
+Before configure add:
+```bash
+	if [[ -z $SKIP_STEP || $SKIP_STEP == "autoconf" ]]; then #not empty allowed as if we bootstrapped above we dont need to run nautoconf
+		gnulib_ensure_buildaux_scripts_copied
+		autoreconf --symlink --verbose --install
+		libtool_fixes
+		autoreconf --verbose #update for libtool fixes
+		SKIP_STEP=""
+	fi
+```
+
+## gzip
+### Template Script Args
+`--BUILD_NAME gzip --HaveOurPatch=0 --GitRepo https://git.savannah.gnu.org/git/gzip.git --BUILD_MSVC_RUNTIME_INFO_ADD_TO_C_AND_LDFLAGS=1`
+
+## rsync
+### Template Script Args
+`--BUILD_NAME rsync --HaveOurPatch=0 --GitRepo https://github.com/WayneD/rsync.git  --GNU_LIBS_ADD_TO_REPO 1  --CONFIG_CMD_ADDL --enable-lz4 --disable-md2man --OUR_LIB_DEPS openssl zstd --VCPKG_DEPS "xxhash" "lz4" --GNU_LIBS_ADDL "getsockopt" "strcase" "strerror" "getaddrinfo" "setsockopt" "sleep" "getsockname" "getpeername" "ioctl" "alloca" "alloca-opt" "socket" "bind" "symlink" "unistd" "fsync" "gettimeofday" "sys_socket" "lock" "flock" "signal-h" "sys_ioctl" "symlink" "symlinkat" "unlinkat" "netinet_in" "arpa_inet" "dirent" "sys_stat" "sys_types" "sys_file" "stdbool" "stat-time" "dirname" "attribute" "dirfd" "dup2" "readlink" "stat-macros" "lstat" "stat-size" "stat-time" "open" "openat" "stdopen" "fcntl" "fcntl-h" "errno"`
+
+after clone
+```bash
+	cp gnulib/build-aux/bootstrap .
+	cp gnulib/build-aux/bootstrap.conf .
+	#it doesn't use extras so we can just ad ours, they use paxutils to gnulib everyhting
+	echo "gnulib_tool_option_extras=\" --without-tests --symlink\"" >> bootstrap.conf
+```
diff --git a/repo_notes/gnutls_README.md b/repo_notes/gnutls_README.md
@@ -4,3 +4,4 @@
 
 - Standard GNULIB patches (though minimal gnulib additional includes due to LGPL/GPL licensing reqs)
 - Support for MSVC, by default only gcc/mingw is supported removed dynamic array from groups
+- Needed to build libtasn1 and p11-kit as well to get complete successful build.  [P11-kit Changes](https://github.com/mitchcapper/p11-kit/compare/master...win32_enhancements).
Original file line number	Diff line number	Diff line change
Expand Up		@@ -4,3 +4,4 @@

		- Standard GNULIB patches (though minimal gnulib additional includes due to LGPL/GPL licensing reqs)
		- Support for MSVC, by default only gcc/mingw is supported removed dynamic array from groups
		- Needed to build libtasn1 and p11-kit as well to get complete successful build. [P11-kit Changes](https://github.com/mitchcapper/p11-kit/compare/master...win32_enhancements).