latch register not working with Docker update

[mjsteinbaugh]

[REQUIRED] Describe the bug

Latch workflow registration is failing due to Docker configuration.

[REQUIRED] To Reproduce

Seeing this on macOS with current Docker, configured to run either at /usr/local/bin (system) or ~/.docker/bin/docker (user).

latch --version
# latch, version 2.19.6
docker --version
# Docker version 23.0.5, build bc4487a
latch register fetch-and-run

❯ latch register fetch-and-run
Unable to register workflow. - printing traceback:

/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/urllib3/connectionpool.py:703 in urlopen
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/urllib3/connectionpool.py:398 in _make_request
/opt/koopa/app/python3.10/3.10.11/lib/python3.10/http/client.py:1283 in request
/opt/koopa/app/python3.10/3.10.11/lib/python3.10/http/client.py:1329 in _send_request
/opt/koopa/app/python3.10/3.10.11/lib/python3.10/http/client.py:1278 in endheaders
/opt/koopa/app/python3.10/3.10.11/lib/python3.10/http/client.py:1038 in _send_output
5 frames collapsed...

--Traceback (most recent call last)--
/opt/koopa/app/python3.10/3.10.11/lib/python3.10/http/client.py:976 in send
  974|         if self.sock is None:
  975|             if self.auto_open:
> 976|                 self.connect()
  977|             else:
  978|                 raise NotConnected()
  979|
  980|         if self.debuglevel > 0:

--Traceback (most recent call last)--
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/docker/transport/unixconn.py:30 in connect
  28|         sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
  29|         sock.settimeout(self.timeout)
> 30|         sock.connect(self.unix_socket)
  31|         self.sock = sock
  32|
  33|     def putheader(self, header, *values):
  34|         super().putheader(header, *values)

FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/requests/adapters.py:486 in send
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/urllib3/connectionpool.py:787 in urlopen
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/urllib3/util/retry.py:550 in increment
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/urllib3/packages/six.py:769 in reraise
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/urllib3/connectionpool.py:703 in urlopen
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/urllib3/connectionpool.py:398 in _make_request
/opt/koopa/app/python3.10/3.10.11/lib/python3.10/http/client.py:1283 in request
/opt/koopa/app/python3.10/3.10.11/lib/python3.10/http/client.py:1329 in _send_request
/opt/koopa/app/python3.10/3.10.11/lib/python3.10/http/client.py:1278 in endheaders
/opt/koopa/app/python3.10/3.10.11/lib/python3.10/http/client.py:1038 in _send_output
9 frames collapsed...

--Traceback (most recent call last)--
/opt/koopa/app/python3.10/3.10.11/lib/python3.10/http/client.py:976 in send
  974|         if self.sock is None:
  975|             if self.auto_open:
> 976|                 self.connect()
  977|             else:
  978|                 raise NotConnected()
  979|
  980|         if self.debuglevel > 0:

--Traceback (most recent call last)--
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/docker/transport/unixconn.py:30 in connect
  28|         sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
  29|         sock.settimeout(self.timeout)
> 30|         sock.connect(self.unix_socket)
  31|         self.sock = sock
  32|
  33|     def putheader(self, header, *values):
  34|         super().putheader(header, *values)

ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/docker/api/client.py:214 in _retrieve_server_version
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/docker/api/daemon.py:181 in version
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/docker/utils/decorators.py:46 in inner
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/docker/api/client.py:237 in _get
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/requests/sessions.py:600 in get
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/requests/sessions.py:587 in request
5 frames collapsed...

--Traceback (most recent call last)--
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/requests/sessions.py:701 in send
  699|
  700|         # Send the request
> 701|         r = adapter.send(request, **kwargs)
  702|
  703|         # Total elapsed time of the request (approximately)
  704|         elapsed = preferred_clock() - start
  705|         r.elapsed = timedelta(seconds=elapsed)

--Traceback (most recent call last)--
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/requests/adapters.py:501 in send
  499|
  500|         except (ProtocolError, OSError) as err:
> 501|             raise ConnectionError(err, request=request)
  502|
  503|         except MaxRetryError as e:
  504|             if isinstance(e.reason, ConnectTimeoutError):
  505|                 # TODO: Remove this in 3.0.0: see #2811

ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/latch_cli/centromere/utils.py:162 in _construct_dkr_client

--Traceback (most recent call last)--
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/docker/api/client.py:197 in __init__
  195|                                 str
  196|                                 ) and version.lower() == 'auto'):
> 197|             self._version = self._retrieve_server_version()
  198|         else:
  199|             self._version = version
  200|         if not isinstance(self._version, str):
  201|             raise DockerException(

--Traceback (most recent call last)--
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/docker/api/client.py:221 in _retrieve_server_version
  219|             )
  220|         except Exception as e:
> 221|             raise DockerException(
  222|                 f'Error while fetching server API version: {e}'
  223|             )
  224|
  225|     def _set_request_timeout(self, kwargs):

DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

/opt/koopa/app/latch/2.19.6/libexec/bin/latch:8 in <module>
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/click/core.py:1130 in __call__
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/click/core.py:1055 in main
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/click/core.py:1657 in invoke
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/click/core.py:1404 in invoke
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/click/core.py:760 in invoke
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/latch_cli/main.py:106 in register
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/latch_cli/services/register/register.py:235 in register
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/latch_cli/centromere/ctx.py:155 in __init__
8 frames collapsed...

--Traceback (most recent call last)--
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/latch_cli/centromere/ctx.py:152 in __init__
  150|                 )
  151|             else:
> 152|                 self.dkr_client = _construct_dkr_client()
  153|         except Exception as e:
  154|             self.cleanup()
  155|             raise e
  156|

--Traceback (most recent call last)--
/opt/koopa/app/latch/2.19.6/libexec/lib/python3.10/site-packages/latch_cli/centromere/utils.py:164 in _construct_dkr_client
  162|             return docker.APIClient(base_url="unix://var/run/docker.sock")
  163|         except docker.errors.DockerException as de:
> 164|             raise OSError(
  165|                 "Docker is not running. Make sure that"
  166|                 " Docker is running before attempting to register a workflow."
  167|             ) from de
  168|

OSError: Docker is not running. Make sure that Docker is running before attempting to register a workflow.

>> Crash report written to .latch_report.tar.gz <<
[1]    68772 segmentation fault  latch register fetch-and-run

[REQUIRED] Expected behavior
A clear and concise description of what you expected to happen.

Workflow should register successfully -- this worked with previous versions of the SDK.

[REQUIRED] Platform Information
General information about host platform + SDK

• OS: macOS Ventura 13.3.1 (a)
• Browser: NA
• Latch Version: 2.19.6

Best,
Mike

[mjsteinbaugh]

Yeah this is a Docker issue with 23.0.5 not working by default with Python. Changing these settings fixes latch register for me:

In Docker Desktop > Settings > Advanced

• Set to System, which will link Docker at /usr/local/bin/docker.
• Check "Enable default Docker socket".
• Check "Enable privileged port mapping".

I think the key setting is to enable "Enable default Docker socket", which will create /var/run/docker.sock, which looks to be required in Python.

[mjsteinbaugh]

For reference, Docker config on macOS is saved in JSON format at: "${HOME}/Library/Group Containers/group.com.docker/settings.json".