Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ pom.xml.asc
/.nrepl-port
/.idea
/*.iml
.cpcache
72 changes: 53 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,14 @@ and then transforms, validates and summarizes it.
*http.clj:*
```clj
(ns my.http
(:require [cyrus-config.core :as cfg]))
(:require [cyrus-config.core :as cfg]
[clojure.spec.alpha :as s]))

;; Introduce a configuration constant that will contain validated and transformed value from the environment
;; By default uses variable name transformed from the defined name: "HTTP_PORT"
(cfg/def HTTP_PORT "Port to listen on" {:spec int?

(s/def ::port int?)
(cfg/def HTTP_PORT "Port to listen on" {:spec ::port
:default 8080})

;; Available immediately, without additional loading commands, but can contain a special value indicating an error.
Expand Down Expand Up @@ -91,12 +94,12 @@ This metadata is used in `(cfg/show)`.
HTTP_PORT
=> 8080
(meta #'HTTP_PORT)
=> {::cfg/user-spec {:spec #object[clojure.core$int_QMARK___5132 ...]}
=> {::cfg/user-spec {:spec :my.http/port}
::cfg/effective-spec {:required true
:default nil
:secret false
:var-name "HTTP_PORT"
:spec #object[clojure.core$int_QMARK___5132 ...]}
:spec :my.http/port}
::cfg/source :environment
::cfg/raw-value "8080"
::cfg/error nil
Expand Down Expand Up @@ -129,8 +132,7 @@ will silently get a special value that indicates an error, for example:

```
* `:default` — default value to use if the variable is not set. Cannot be used together with `:required true`. Defaults to `nil`.
* `:spec` — Clojure Spec to conform the value to. Defaults to `string?`, can also be `int?`, `keyword?`, `double?` and
any complex spec, in which case the original value will be parsed as EDN and then conformed. See Conforming/Coercing section below.
* `:spec` — Clojure Spec to conform the value to. Defaults to string. Specs must be provided as fully-qualified registered keywords. See Conforming/Coercing section below.
* `:schema` — Prismatic Schema to coerce the value to (same as `:spec`, but for Prismatic). Complex schemas first parse the value as YAML.
* `:secret` — boolean, if true, the value will not be displayed in the overview returned by `(cfg/show)`:
```
Expand All @@ -140,10 +142,11 @@ will silently get a special value that indicates an error, for example:
You can also use existing configuration constants' values when defining configuration constants:

```clj
(s/def ::server-polling-interval int?)
(cfg/def SERVER_URL)

;; This constant will only be required if SERVER_URL is set
(cfg/def SERVER_POLLING_INTERVAL {:required (some? SERVER_URL) :spec int?})
(cfg/def SERVER_POLLING_INTERVAL {:required (some? SERVER_URL) :spec ::server-polling-interval})

;; This will get default value from SERVER_POLLING_INTERVAL, when it's set (it also has a different type)
(cfg/def SERVER_POLLING_DELAY {:default SERVER_POLLING_INTERVAL})
Expand All @@ -163,7 +166,8 @@ The output looks like this:
cyrus-config.core/validate! core.clj: 146
clojure.core/ex-info core.clj: 4739
clojure.lang.ExceptionInfo: Errors found when loading config:
#'my.http/HTTP_PORT: <ERROR> because HTTP_PORT contains "abcd" in :environment - java.lang.NumberFormatException: For input string: "abcd" // Port to listen on

#'my.http/HTTP_PORT: <ERROR> because HTTP_PORT contains "abcd" in :environment - clojure.lang.ExceptionInfo: Failed to coerce value {:spec :my.http/port, :value "abcd"} // Port to listen on
```

#### Summary
Expand Down Expand Up @@ -217,20 +221,26 @@ This will ensure that every time the code is reloaded, the overrides file `dev-e
### Conforming/Coercion

The library supports two ways of conforming (a.k.a. coercing) environment values (which are always string) to
various types: integer, keyword, double, etc. The ways of defining targer types are Clojure Spec and Prismatic Schema.
various types: integer, keyword, double, etc. The ways of defining target types are Clojure Spec and Prismatic Schema.
They are mutually exclusive, i.e. only one of `:spec` and `:schema` keys are possible at the same time for each configuration constant.

#### Clojure Spec

> [spec] is a Clojure library to describe the structure of data and functions.

It is enabled by setting `:spec` key in the parameters:
The Clojure Spec coercion implementation is provided by [spec-coerce](https://github.com/wilkerlucio/spec-coerce). It is enabled by setting `:spec` key in the parameters:

```clj
(cfg/def HTTP_PORT {:spec int?})
(s/def ::port int?)
(cfg/def HTTP_PORT {:spec ::port})
```

Implicit coercion is in place for basic types: `int?`, `double?`, `boolean?`, keyword?`, `string?` (the default one, does nothing).
Implicit coercion is in place for basic types: `int?`, `double?`, `boolean?`, `keyword?`, `string?` (the default one does nothing). But note that you must provide the spec in the form of a registered, fully-qualified keyword as defined by `s/def`. It's expected that you will be defining these on your own (as in the example of `::port` above), but if you'd rather not, there are some stand-in specs for the basic types located under `cyrus-config.coerce`:
- :cyrus-config.coerce/int
- :cyrus-config.coerce/double
- :cyrus-config.coerce/keyword
- :cyrus-config.coerce/string
- :cyrus-config.coerce/boolean

##### Custom coercions

Expand Down Expand Up @@ -258,10 +268,15 @@ Additionally, you can put a complex value in EDN format into the variable:

IP_WHITELIST='["1.2.3.4" "4.3.2.1"]'

and then conform it:
and then define how to coerce it using spec-coerce's `def` form:

```clj
(cfg/def IP_WHITELIST {:spec (cfgc/from-edn (s/coll-of string?))})
(require '[spec-coerce.core :as sc]
'[clojure.spec.alpha :as s]
'[clojure.edn :as edn])
(s/def ::ip-whitelist (s/coll-of string?))
(sc/def ::ip-whitelist edn/read-string)
(cfg/def IP_WHITELIST {:spec ::ip-whitelist})
IP_WHITELIST
=> ["1.2.3.4" "4.3.2.1"]
;; ^ not a string, a Clojure data structure
Expand All @@ -272,7 +287,8 @@ Alternatively, you can use JSON format:
IP_WHITELIST='["1.2.3.4", "4.3.2.1"]'

```clj
(cfg/def IP_WHITELIST {:spec (cfgc/from-custom-parser json/parse-string (s/coll-of string?))})
(require '[cheshire.core :as json])
(sc/def ::ip-whitelist json/parse-string)
```

Or any other custom conversion:
Expand All @@ -286,17 +302,35 @@ Or any other custom conversion:
(->> (str/split (str csv) #",")
(map str/trim))))

(cfg/def IP_WHITELIST {:spec (s/conformer parse-csv)})
(sc/def ::ip-whitelist parse-csv)
```

In this case, conversion is considered successful if it does not throw an exception.

`if (sequential? csv)` condition is important, it allows to provide `:default` not only as string, but also as target type:

```clj
(cfg/def IP_WHITELIST {:spec (s/conformer parse-csv) :default ["one" "two"]})
(cfg/def IP_WHITELIST {:spec ::ip-whitelist :default ["one" "two"]})
```

Note also that spec-coerce honors the "first" or "left-most" spec condition of `s/and` specs for coercion. So, when using one of the basic predicates (like `keyword?` or `int?`) for coercing, you can specify the general coercion spec first and then the more granular constraints afterward to avoid the need for an `sc/def` line:

```clj
(s/def ::my-number (s/and int? (s/int-in 0 2)))
;; Is the same as:
(s/def ::my-number (s/int-in 0 2))
(sc/def ::my-number sc/parse-long)
```

See [spec-coerce's usage](https://github.com/wilkerlucio/spec-coerce#usage) for more details and options.

NOTE: Taking this approach may hamper your ability to generate sample data from these specs. `::my-number` will sometimes fail to generate after 100 tries, because the `int?` case is very broad and the `s/int-in` case is very narrow. It seems spec's generator logic also reads left-to-right in this sense.

A better solution to the specific `::my-number` example above would be to use a homogeneous set:

```clj
(s/def ::my-number #{0 1})
```

spec-coerce will infer from this set that it needs to parse as an integer.

#### Prismatic Schema

Expand Down
11 changes: 11 additions & 0 deletions deps.edn
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{:deps {org.clojure/clojure {:mvn/version "1.10.0"}
squeeze {:mvn/version "0.3.3"}
spec-coerce {:mvn/version "1.0.0-alpha14"}}
:aliases {:dev {:extra-deps {cheshire {:mvn/version "5.8.1"}}}
:test {:extra-paths ["test"]
:extra-deps {cheshire {:mvn/version "5.8.1"}
com.cognitect/test-runner
{:git/url "[email protected]:cognitect-labs/test-runner"
:sha "76568540e7f40268ad2b646110f237a60295fa3c"}}
:main-opts ["-m" "cognitect.test-runner"]}}
:paths ["src"]}
2 changes: 1 addition & 1 deletion project.clj
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
:url "https://github.com/dryewo/cyrus-config"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies []
:dependencies [[spec-coerce "1.0.0-alpha14"]]
:plugins [[lein-cloverage "1.0.13"]
[lein-shell "0.5.0"]
[lein-ancient "0.6.15"]
Expand Down
69 changes: 9 additions & 60 deletions src/cyrus_config/coerce.clj
Original file line number Diff line number Diff line change
@@ -1,78 +1,27 @@
(ns cyrus-config.coerce
(:require [clojure.spec.alpha :as s]
[clojure.edn :as edn]
[clojure.string :as str]))

[clojure.string :as str]
[spec-coerce.core :as sc]))

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Custom parsers


(defn wrapped-string-parser [parser-fn]
(fn [data]
(if (string? data)
(try
(parser-fn data)
(catch Exception _ ::s/invalid))
data)))


(defmacro from-custom-parser [parser-fn spec]
`(s/and
(s/spec-impl '~parser-fn (wrapped-string-parser ~parser-fn) nil true)
~spec))


(defn from-edn [spec]
(from-custom-parser edn/read-string spec))


(defn unblank [s]
(let [str-s (str s)]
(when-not (str/blank? str-s)
str-s)))

(s/def ::nonblank-string (s/nilable string?))
(sc/def ::nonblank-string unblank)

(s/def ::nonblank-string (s/conformer unblank))

(s/def ::int int?)
(s/def ::double double?)
(s/def ::keyword keyword?)
(s/def ::string string?)
(s/def ::boolean boolean?)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;


(defn wrapped-parser [spec parser-fn]
(fn [data]
(if (s/valid? spec data)
data
(try
(parser-fn (str data))
(catch Exception _ ::s/invalid)))))


;; TODO make it look pretty: {int? 'Integer/parseInt double? 'Double/parseDouble ...}
(def known-parsers
{int? (s/spec-impl 'Integer/parseInt (wrapped-parser int? #(Integer/parseInt %)) nil true)
double? (s/spec-impl 'Double/parseDouble (wrapped-parser double? #(Double/parseDouble %)) nil true)
boolean? (s/spec-impl 'Boolean/parseBoolean (wrapped-parser boolean? #(Boolean/parseBoolean %)) nil true)
keyword? (s/spec-impl 'keyword (wrapped-parser keyword? #(keyword %)) nil true)
string? (s/conformer str)})


(defn conformer-for-spec [spec]
(get known-parsers spec spec))


(defn coerce-to-spec [spec data]
(let [spec-with-conformer (conformer-for-spec spec)
result (s/conform spec-with-conformer data)]
(if (= result ::s/invalid)
(throw (Exception. (str "Error coercing " (pr-str data) ": "
(str/trim-newline (s/explain-str spec-with-conformer data)))))
result)))


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;


(def ^:private squeeze-coerce-to-schema
(try
(require 'squeeze.core)
Expand Down
6 changes: 4 additions & 2 deletions src/cyrus_config/core.clj
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
(ns cyrus-config.core
(:require [clojure.spec.alpha :as s]
[clojure.string :as str]
[spec-coerce.core :as sc]
[cyrus-config.coerce :as c])
(:import (java.io Writer)
(java.util LinkedHashSet)
Expand Down Expand Up @@ -38,6 +39,7 @@
(s/def ::secret boolean?)
(s/def ::var-name (s/or :string string? :keyword keyword?))
(s/def ::config-definition (s/keys :opt-un [::spec ::schema ::required ::default ::secret ::var-name]))
(s/def ::default-coercion-spec string?)

(s/fdef effective-config-definition
:args (s/cat :name symbol? :definition ::config-definition))
Expand Down Expand Up @@ -77,11 +79,11 @@
(try
(cond
spec
[(c/coerce-to-spec spec raw-value)]
[(sc/coerce! spec raw-value)]
schema
[(c/coerce-to-schema schema raw-value)]
:else
[(c/coerce-to-spec string? raw-value)])
[(sc/coerce! ::default-coercion-spec raw-value)])
(catch Exception e
(let [error {:code ::invalid-value :value raw-value :message (str e)}]
[(ConfigNotLoaded. error) error])))))]
Expand Down
Loading