Skip to content

Autoscaler is failing with KeyError: 'headnode.internal.cloudapp.net' #85

@xpillons

Description

@xpillons

On some systems, it appears that PBS commands qmgr list sched and qmgr list server sometimes return values with long names and not short names, making the research of a scheduler to failed with the following stack trace

2025-09-18 08:49:02,300 DEBUG: Full stacktrace Traceback (most recent call last): File "/opt/cycle/pbspro/venv/lib/python3.6/site-packages/hpc/autoscale/clilib.py", line 1890, in main args.func(**kwargs) File "/opt/cycle/pbspro/venv/lib/python3.6/site-packages/hpc/autoscale/clilib.py", line 434, in autoscale output_columns = output_columns or self._get_default_output_columns(config) File "/opt/cycle/pbspro/venv/lib/python3.6/site-packages/hpc/autoscale/clilib.py", line 1075, in _get_default_output_columns default_cmd = self._default_output_columns(config, cmd_name) File "/opt/cycle/pbspro/venv/lib/python3.6/site-packages/pbspro/cli.py", line 76, in _default_output_columns env = self._pbs_env(driver) File "/opt/cycle/pbspro/venv/lib/python3.6/site-packages/pbspro/cli.py", line 110, in _pbs_env self.__pbs_env = environment.from_driver(pbs_driver.config, pbs_driver) File "/opt/cycle/pbspro/venv/lib/python3.6/site-packages/pbspro/environment.py", line 52, in from_driver schedulers = pbs_driver.read_schedulers() File "/opt/cycle/pbspro/venv/lib/python3.6/site-packages/pbspro/driver.py", line 562, in read_schedulers return read_schedulers(self.pbscmd, self.resource_definitions) File "/opt/cycle/pbspro/venv/lib/python3.6/site-packages/pbspro/scheduler.py", line 84, in read_schedulers server_dict = server_dicts_by_host[hostname] KeyError: 'headnode.internal.cloudapp.net'

A suggested fix would be to use short names in the dictionary lookup line 83 and make sure that the scheduler is always using short names.

Image

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions