Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(processor): replace filepath.Glob with doublestar.Glob for enhanced pattern matching #99

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

refactor(processor): replace filepath.Glob with doublestar.Glob for enhanced pattern matching #99

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
299de17
refactor(processor): replace filepath.Glob with doublestar.Glob for e…
Mar 3, 2025
ea5b98e
docs: add documentation for unix-like glob path pattern support in mo…
Mar 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/getting-started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,11 @@ $ ./output/modctl -h

Build the model artifact you need to prepare a Modelfile describe your expected layout of the model artifact in your model repo.

Notes: We support Unix-like glob path patterns, where:
- `*` matches any number of characters except path separators (`/`).
- `**` matches any number of directories, including zero directories.
For example, `**/*.py` matches all `.py` files in the current directory and its subdirectories.

Example of Modelfile:

```shell
Expand Down
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ require (
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bmatcuk/doublestar/v4 v4.8.1
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bmatcuk/doublestar/v4 v4.8.1 h1:54Bopc5c2cAvhLRAzqOGCYHYyhcDHsFF4wWIR5wKP38=
github.com/bmatcuk/doublestar/v4 v4.8.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
Expand Down
9 changes: 7 additions & 2 deletions pkg/backend/processor/base.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,15 @@ package processor
import (
"context"
"fmt"
"os"
"path/filepath"
"sort"
"sync"
"sync/atomic"

"github.com/CloudNativeAI/modctl/pkg/backend/build"
"github.com/CloudNativeAI/modctl/pkg/storage"
doublestar "github.com/bmatcuk/doublestar/v4"

"github.com/chelnak/ysmrr"
humanize "github.com/dustin/go-humanize"
Expand Down Expand Up @@ -58,11 +60,14 @@ func (b *base) Process(ctx context.Context, workDir, repo string, opts ...Option

var matchedPaths []string
for _, pattern := range b.patterns {
matches, err := filepath.Glob(filepath.Join(absWorkDir, pattern))
matches, err := doublestar.Glob(os.DirFS(absWorkDir), pattern)
if err != nil {
return nil, err
}

// convert to absolute paths
for i := range matches {
matches[i] = filepath.Join(absWorkDir, matches[i])
}
matchedPaths = append(matchedPaths, matches...)
}

Expand Down