[FEATURE REQUEST] Generate OCClasseSettings documentation automatically

[mneuwert]

UPDATE: The part regarding the AppConfig spec file has been split out and moved to #842.

---

Is your feature request related to a problem? Please describe.
In ownCloud app we have a comprehensive set of settings which can be used for branding, MDM, configuration during development. Documentation as well as AppConfig spec files are maintained manually. When new setting is added in the code, I think it is not really ensured that all other artefacts are updated accordingly.

Describe the solution you'd like
Would be nice to be able to extract OCClassSetting information from the code for example using Apple's SourceKit framework. There is also interesting framework / command line tool which abstracts SourceKit called SourceKitten - https://github.com/jpsim/SourceKitten which e.g. can create an XML file describing structure from an Objective-C source file.

So, we could create a small tool run by a custom build phase in Xcode which could generate the artefacts we need.

May be we would need to augment the definition of our settings by adding special comments describing the setting and e.g. also adding tags: mandatory, nice-to-have, supported, development etc.

TASKS

• Research (if needed)
• Create branch feature/feature_name
• Development tasks
  • Implement whatever
  • ...
• Code review and apply changes requested
• Design test plan
• QA
• Merge branch feature/feature_name into master

PR

• App
• Library (if needed)

[michaelstingl]

For every config key/parameter, we'd need some metadata:

• Configuration Key
• Type (Boolean, Integer, String, String Array, …)
• Description
• Possible values
• Default value
• Category
• Status (Recommended, Approved for production, Debugging-only…)

@felix-schwarz @mneuwert would it be feasible to make every parameter like an "array" or JSON like structure? It hould already contain the information for the AppConfig, maybe some more metadata to render in our docs. How could the syntax look like? (for example for the passcode.enforced parameter)

ios-app/docs/modules/ROOT/pages/ios_mdm.adoc

Lines 177 to 191 in 0808ad9

	==== Passcode Enforcement
	If your organization policies require users to use a passcode as an additional security barrier for managed apps, the below setting will allow to enforce this requirement.
	[cols=4*,options=header]
	|===
	|Key
	|Type
	|Default
	|Description
	|passcode.enforced
	|Boolean
	|`false`
	|If set to true, the app will require the user to set up a passcode upon a first launch.
	|===

Then it should be easier to parse nice docs and XML files?

"Category" could be used for the grouping parameters of similar topic.

Only config parameters with "Recommended" status should be included in an XML file that gets submitted to EMM vendors…

[michaelstingl]

MDM options can be autogenerated as JSON now:
https://github.com/owncloud/ios-sdk/blob/feature/settings-metadata/doc/CONFIGURATION.json
(will replaced by bigger JSON in repo ios-app later)

@dschmidt will help with gomplate to create nice AsciiDoc tables for docs, similar to what we have today:
https://github.com/owncloud/ios-app/blob/feature/mdm_enhancements/docs/modules/ROOT/pages/ios_mdm.adoc

I was thinking, individual AsciiDoc tables could be generated per category ("category" : "Logging",), then embedded in the big AsciiDoc page…

Entries should be sorted by status ("status" : "supported",):

1. "status" : "supported",
2. "status" : "advanced",
3. "status" : "debugOnly", (maybe hidden)

Table columns needed:

1. Key
2. Type
3. Default value
4. Description
5. Status

gomplate could run on local developers machine, and updated AsciiDoc tables will be part of commit/PR.

[dschmidt]

I have prepared the gomplate template, but the format of CONFIGURATION.json is incompatible, because gomplate does not like a single \ for escaping forward slashes. Replacing all occurrences of \/ with / makes it work.

I noticed some configuration items are present multiple times, e.g. vacuum-sync-anchor-ttl. IMHO that should be fixed in CONFIGURATION.json.

The template looks as follows:

{{- $config := .config }}
{{- $categories := .config | jsonpath `$[*].category` | uniq | sort }}
{{- range $index, $category := $categories }}
tag::{{$category | strings.ToLower }}[]
[cols=4*,options=header] 
|=== 
|Key 
|Type 
|Default 
|Description 

{{$options := $config | jsonpath (print "$[?(@.category=='" $category "')]") }}
{{- if eq (printf "%T" $options) "map[string]interface {}"}}
{{- $option := $options }}
|{{- $option.key}}
|{{- $option.type}} 
|{{- if has $option "defaultValue"}}`{{$option.defaultValue}}`{{end}} 
|{{- if has $option "description"}}{{$option.description}}{{end}}
{{- else}}
{{- range $statusIndex, $status := slice "supported" "advanced" "debugOnly"}}
{{- range $optionIndex, $option := $options | jsonpath (print "$[?(@.status=='" $status "')]")}}
|{{- $option.key}}
|{{- $option.type}} 
|{{- if has $option "defaultValue"}}`{{$option.defaultValue}}`{{end}} 
|{{- if has $option "description"}}{{$option.description}}{{end}}
{{end}}
{{- end}}
{{- end}}
|=== 
end::{{$category | strings.ToLower }}[]

{{end}}

The actual .adoc file can be generated via

../ios-app/docs/modules/ROOT/pages $ gomplate -f ios_mdm_tables.adoc.tmpl --context config=../../../../CONFIGURATION.json -o ios_mdm_tables.adoc

... and afterwards tables can be referenced in the main .adoc file via:

== Authentication
include::ios_mdm_tables.adoc[tag=authentication] 

== Connection
include::ios_mdm_tables.adoc[tag=connection]

== Endpoints
include::ios_mdm_tables.adoc[tag=endpoints]

== Logging
include::ios_mdm_tables.adoc[tag=logging]

== OAuth2
include::ios_mdm_tables.adoc[tag=oauth2]

== OIDC
include::ios_mdm_tables.adoc[tag=oidc]

== Policies
include::ios_mdm_tables.adoc[tag=policies]

== Privacy
include::ios_mdm_tables.adoc[tag=privacy]

== Security
include::ios_mdm_tables.adoc[tag=security]

I can prepare a PR for adding it to the repo, as soon as a compatible CONFIGURATION.json is added to this repo.
A prepared place to hook gomplate into would be appreciated.

P.S.: this needs gomplate >= 3.8.0, otherwise it won't work.

[michaelstingl]

@dschmidt very cool! 💪

I was able to generate a table from the new complete JSON:
https://github.com/owncloud/ios-app/blob/feature/settings-metadata/docs/modules/ROOT/pages/ios_mdm_tables.adoc

Please add 2 more columns:

• Possible values
• Status

[felix-schwarz]

@dschmidt Thanks, that's awesome already!

I have prepared the gomplate template, but the format of CONFIGURATION.json is incompatible, because gomplate does not like a single \ for escaping forward slashes. Replacing all occurrences of \/ with / makes it work.

I have changed the generator code to avoid these. The CONFIGURATION.json no longer escapes the slashes.

I noticed some configuration items are present multiple times, e.g. vacuum-sync-anchor-ttl. IMHO that should be fixed in CONFIGURATION.json.

Good catch. I fixed that as well.

I can prepare a PR for adding it to the repo, as soon as a compatible CONFIGURATION.json is added to this repo.
A prepared place to hook gomplate into would be appreciated.

I'll put together a bash script that generates the JSON documentation, to which gomplate could also be added then.

Regarding possible values, I also made a change to the JSON format that makes it easier to convert. Previously, possible values could come in either of these formats:

    "possibleValues" : {
      "-1" : "verbose",
      "0" : "debug",
      "1" : "info",
      "2" : "warning",
      "3" : "error",
      "4" : "off"
    }

… or more verbose:

    "possibleValues" : [
      {
        "description" : "debug",
        "value" : 0
      },
      {
        "description" : "error",
        "value" : 3
      },
      {
        "description" : "off",
        "value" : 4
      },
      {
        "description" : "warning",
        "value" : 2
      },
      {
        "description" : "verbose",
        "value" : -1
      },
      {
        "description" : "info",
        "value" : 1
      }
    ],

The generated JSON now always uses the latter, verbose format.

Besides normalizing possibleValues to a single format, it also preserves the type more closely (for the more compact format, the values always needed to be converted to strings).

[felix-schwarz]

@dschmidt I put together a script here now: https://github.com/owncloud/ios-app/blob/feature/settings-metadata/tools/GenerateDocs/generate_docs.sh

And also put your template here: https://github.com/owncloud/ios-app/blob/feature/settings-metadata/tools/GenerateDocs/templates/ios_mdm_tables.adoc.tmpl

When running the script from its folder, it runs gomplate with

gomplate -f templates/ios_mdm_tables.adoc.tmpl --context config=../../doc/CONFIGURATION.json -o ../../docs/modules/ROOT/pages/ios_mdm_tables.adoc

which partially produces the documentation, then fails with this error:

16:55:52 FTL  error="template: templates/ios_mdm_tables.adoc.tmpl:22:12: executing \"templates/ios_mdm_tables.adoc.tmpl\" at <$option.key>: can't evaluate field key in type interface {}"

[dschmidt]

I took the liberty to push to your branch:

• d266176#diff-d5f4c9ac6c58d3c3c6c0bc1e7e11fd4a3e50d8faba23e0b7d505bd8dbbbaf067
  this is the actual fix. The problem was that now (maybe after your fixes) there were categories with only one item of a specific status, that made jsonpath return a single item instead of a list (:sigh:) which wreaked havoc ...
• d6d776b#diff-d5f4c9ac6c58d3c3c6c0bc1e7e11fd4a3e50d8faba23e0b7d505bd8dbbbaf067
  reduces the duplication for single items. I had tried to use the very same approach in my initial implementation but maybe it was too late on that day or something ... but now it works, yay.
• cad60c0#diff-d5f4c9ac6c58d3c3c6c0bc1e7e11fd4a3e50d8faba23e0b7d505bd8dbbbaf067
  Added status column

How should possible values be rendered in the table?

[michaelstingl]

How should possible values be rendered in the table?

For me, just listing them inline would do the job…
"-1" : "verbose", "0" : "debug", "1" : "info", "2" : "warning", "3" : "error", "4" : "off"

@felix-schwarz does this make sense for you?

[felix-schwarz]

@dschmidt Thanks!

@michaelstingl Personally, I'd prefer a table with possible values in the description column below the description - provided that's possible:

`log-level`

integer

Controls how much details are logged. Possible values Value Description -1 Verbose 0 Debug 1 Info 2 Warning 3 Error 4 Off

We could have more than one word as description in the future, in which case that formatting would provide a better / easier overview.

[michaelstingl]

I'd prefer a table with possible values in the description column below the description

Fine for me. @dschmidt some refinement next week?

[dschmidt]

6551b2b
->
https://github.com/owncloud/ios-app/blob/6551b2bc241755a7e56c16b3ef8d8251bab9145f/docs/modules/ROOT/pages/ios_mdm_tables.adoc