Skip to content

Conversation

JonSnow1807
Copy link

Description

This PR adds support for Colima as an alternative to Docker Desktop for running metaflow-dev on macOS.

Note: This is a clean PR containing only the Colima support changes. It supersedes #2519 which accidentally included unrelated commits.

Problem

The current check-docker target in the devtools/Makefile uses open -a Docker which only works with Docker Desktop. This prevents users who use Colima (or other Docker alternatives) from using metaflow-dev.

Error users see:

❌ Please start Docker Desktop

Solution

Modified the Docker detection logic to:

  1. First check if the docker daemon is actually running using docker info
  2. If running, identify which runtime is being used (Colima, Docker Desktop, or unknown)
  3. Update error messages to mention both Docker Desktop and Colima as options

Changes

  • Modified devtools/Makefile check-docker target to support multiple Docker runtimes
  • Replaced macOS-specific open -a Docker with runtime-agnostic docker daemon check
  • Added detection for Colima using colima status
  • Fixed Colima detection by redirecting stderr to stdout (2>&1) as colima outputs to stderr
  • Enhanced user feedback to show which Docker runtime is being used

Testing

Tested on macOS Sequoia (ARM64) with:

  • ✅ Colima 0.8.1 running - correctly detected as "Using Colima as Docker runtime"
  • ✅ No Docker runtime - shows appropriate error message mentioning both options
  • ✅ Docker CLI 28.3.2

Why This Matters

Many organizations cannot use Docker Desktop due to licensing requirements. Colima is a popular open-source alternative. This change removes a significant barrier to Metaflow development for these users.

Backward Compatibility

The changes maintain full backward compatibility:

  • Docker Desktop users see no change in behavior
  • The detection gracefully handles unknown Docker runtimes
  • Linux behavior is unchanged

Fixes #2362

cc: @savingoyal (issue creator)

- Modified check-docker target in devtools/Makefile to support both Docker Desktop and Colima
- Replaced macOS-specific 'open -a Docker' with docker daemon check using 'docker info'
- Added runtime detection to identify whether Colima or Docker Desktop is being used
- Fixed Colima detection by redirecting stderr to stdout (2>&1) as colima outputs to stderr
- Updated error messages to guide users to start either Docker Desktop or Colima
- Maintains full backward compatibility with existing Docker Desktop users

Testing:
- Tested with Colima 0.8.1 on macOS Sequoia (ARM64)
- Verified Colima detection works correctly in full metaflow-dev flow
- Verified error messages when no Docker runtime is available
- Confirmed backward compatibility logic is preserved

Fixes Netflix#2362
@npow
Copy link
Contributor

npow commented Jul 29, 2025

I don't think this PR is complete. Where is it actually using colima? Am I misunderstanding something?

@JonSnow1807
Copy link
Author

@npow Thank you for catching that! You're absolutely right, I misunderstood and only added detection for Colima but didn't actually configure Docker to use it.

I now understand that I need to:

  1. Set the DOCKER_HOST environment variable to point to Colima's socket
  2. Switch the Docker context to use Colima
  3. Make sure all Docker commands in the Makefile use these settings

I'll update the PR with a complete implementation that actually makes Docker commands work through Colima, not just detect it. Give me a bit to implement this properly and I'll push an update.

Thanks for the helpful review!

@JonSnow1807
Copy link
Author

Closing this PR in favor of #2530 which includes the complete implementation with proper Docker context management, environment configuration, setup scripts, and documentation as suggested in the review.

Thank you @npow for the helpful feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support colima for metaflow-dev
2 participants