Configurable USER_CONF_ROOT path

[ScottWales]

Problem

The USER_CONF_ROOT path is used in a variety of places for storing logs, secrets and the Jupyter database. While most uses are configurable in jupyter_config.py, the output path of RotatingUISFileHandler is not. We'd like to place this log in a different location with append-only permissions.

Proposed Solution

• Allow USER_CONF_ROOT to be configured with an environment variable, e.g. $CYLC_CONF_PATH in the same way as cylc-flow.

and/or

• Add a new configuration item e.g. c.CylcUIServer.server_log_path that sets an output path for RotatingUISFileHandler (defaulting to USER_CONF_ROOT/log as it currently does).

I can see a complication that RotatingUISFileHandler is initialised very early in the startup, potentially before the configuration is loaded.

[hjoliver]

Sounds reasonable.

[oliver-sanders]

Note, the logging that Jupyter Server / Cylc UI Server uses is already configurable.

Here's the default configuration:

cylc-uiserver/cylc/uiserver/jupyter_config.py

Lines 67 to 96 in de7580a

	c.CylcUIServer.logging_config = {
	'version': 1,
	'handlers': {
	'file': {
	'class': 'logging.handlers.RotatingFileHandler',
	'level': 'INFO',
	'filename': str(USER_CONF_ROOT / 'log' / 'log'),
	'mode': 'a',
	'backupCount': 5,
	'maxBytes': 10485760,
	'formatter': 'file_fmt',
	},
	},
	'loggers': {
	'CylcUIServer': {
	'level': 'INFO',
	'handlers': ['console', 'file'],
	},
	'cylc': {
	'level': 'INFO',
	'handlers': ['console', 'file'],
	},
	},
	'formatters': {
	'file_fmt': {
	'format': '%(asctime)s %(levelname)-8s %(message)s',
	'datefmt': '%Y-%m-%dT%H:%M:%S',
	}
	},
	}

E.G. swap out the filename for something like Path('~/my-log-dir').expanduser() and put that in the site-config to point all logging somewhere different.

[oliver-sanders]

We'd like to place this log in a different location with append-only permissions.

Unix file permissions are read/write/execute so I think append-only permissions could only be implemented at the filesystem level using something funky.

Note that Cylc is currently configured to open the log file using Python's append mode:

cylc-uiserver/cylc/uiserver/jupyter_config.py

Line 74 in de7580a

'mode': 'a',

[oliver-sanders]

Note, you can do some advanced things like chown using Python's logging config which is what's used to interpret the logging_config configuration e.g:

https://docs.python.org/3/howto/logging-cookbook.html#customizing-handlers-with-dictconfig