diff --git a/CHANGELOG.md b/CHANGELOG.md index 3e7ac85a..76755fbc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -32,6 +32,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Human-specific metadata guidelines (templates/human/README.adoc) - MS data file metadata (in ms-proteomics template) - SDRF terms reference (sdrf-terms.tsv) +- **Template Builder page** (site/sdrf-builder.html): dedicated interactive wizard for building customized SDRF templates by selecting technology, organism, and experiment type. +- **Metaproteomics examples**: PXD005969 (human gut, extraction methods), PXD003572 (soil, Mediterranean dryland), PXD009712 (ocean, Pacific depth profiles). - **Website infrastructure** (site/): homepage, SDRF explorer, terms reference, search functionality, CSS styling. - **PDF generation workflow** with custom theme for specification documents. - File-level metadata support using dedicated columns (`comment[sdrf version]`, `comment[sdrf template]`, `comment[sdrf annotation tool]`) for capturing SDRF version, template, and provenance information. @@ -53,6 +55,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - **Affinity proteomics**: version set to 1.0.0. `comment[instrument]` renamed to `comment[platform]` (REQUIRED); new `comment[instrument]` added as OPTIONAL for actual sequencer/reader. Sample type values normalized to use spaces (`sample control`, `negative control`, etc.). - **Cell-lines**: added PATO ontology to disease field for `normal` (PATO:0000461). Added `characteristics[culture medium]` (RECOMMENDED) and `characteristics[storage temperature]` (RECOMMENDED). - **MS-proteomics/DDA**: mass tolerance patterns updated to accept `not available`/`not applicable` when those flags are set. DDA mass tolerance kept as RECOMMENDED. +- **Quick Start page** refactored as an educational concepts guide (column types, ontology usage, common patterns, validation) with CTA linking to the new Template Builder. +- **Navigation** updated across all pages to include Template Builder link. +- **CI/CD link checker** optimized with concurrency limits, retries, and GITHUB_TOKEN to avoid 429 rate-limit failures. Fixed stale repository URLs (proteomics-sample-metadata → proteomics-metadata-standard). +- **Dev/Stable version links** now use a placeholder system in inject-headers.py, resolved at build time based on `--dev` flag. - **Specification restructured** with clearer organization: Quick Start → Validation → Specification Structure → Notational Conventions → Sample Metadata → Data File Metadata → Templates → Factor Values. - **Column naming**: `fileformat` changed to `file_format` for consistency with underscore convention. - **Ontology recommendations**: added NCIT and PRIDE to general purpose; added PATO for healthy samples (`normal` = PATO:0000461). diff --git a/llms.txt b/llms.txt index a862aacb..654e5ecd 100644 --- a/llms.txt +++ b/llms.txt @@ -69,6 +69,15 @@ Machine-readable YAML definitions used by sdrf-pipelines for validation. Each te - sdrf-proteomics/sdrf-templates/somascan/1.0.0/somascan.yaml - SomaScan (experiment layer): aptamer-based proteomics - sdrf-proteomics/sdrf-templates/somascan/1.0.0/somascan.sdrf.tsv - SomaScan example +## Website Pages + +- site/index.html - Homepage: overview, quick links, templates table, tools, examples, contributors +- site/quickstart.html - Quick Start Guide: SDRF concepts, column types, ontology usage, common patterns, validation +- site/sdrf-builder.html - Template Builder: interactive wizard to select technology, organism, and experiment type and generate a customized SDRF template +- site/sdrf-explorer.html - SDRF Explorer: browse and filter 298+ annotated proteomics datasets +- site/sdrf-editor.html - SDRF Editor: browser-based editor with ontology autocomplete +- site/sdrf-terms.html - SDRF Terms Reference: browsable table of all column terms with ontology mappings + ## Tools - sdrf-proteomics/tool-support.adoc - Tool Support Overview: annotators, validators, analysis tools diff --git a/psi-document/sdrf-proteomics-specification-v1.1.0-dev.pdf b/psi-document/sdrf-proteomics-specification-v1.1.0-dev.pdf index d5236591..683edd7c 100644 Binary files a/psi-document/sdrf-proteomics-specification-v1.1.0-dev.pdf and b/psi-document/sdrf-proteomics-specification-v1.1.0-dev.pdf differ diff --git a/scripts/add-dev-banner.sh b/scripts/add-dev-banner.sh index 319c324e..e11f54a1 100755 --- a/scripts/add-dev-banner.sh +++ b/scripts/add-dev-banner.sh @@ -1,19 +1,37 @@ #!/bin/bash -# Add development version banner to documentation +# Add development version banner to all documentation pages # Usage: ./add-dev-banner.sh set -e OUTPUT_DIR="${1:-docs}" -echo "Adding dev banner to: $OUTPUT_DIR" +echo "Adding dev banner to all HTML pages in: $OUTPUT_DIR" -# Add banner HTML to index.html -sed -i.bak 's//
⚠️ Development Version - This documentation is from the dev branch and may contain unreleased changes. View stable version<\/a><\/div>/' "$OUTPUT_DIR/index.html" -rm -f "$OUTPUT_DIR/index.html.bak" +# Use Python for reliable cross-platform HTML injection +python3 -c " +import os, re -# Append dev banner CSS to stylesheet -cat >> "$OUTPUT_DIR/css/style.css" << 'EOF' +output_dir = '$OUTPUT_DIR' +banner = '
⚠️ Development Version - This documentation is from the dev branch and may contain unreleased changes. View stable version
' + +for root, dirs, files in os.walk(output_dir): + for f in files: + if not f.endswith('.html'): + continue + path = os.path.join(root, f) + with open(path, 'r') as fh: + content = fh.read() + if 'dev-banner' in content: + continue + content = re.sub(r'(]*>)', r'\1' + banner, content) + with open(path, 'w') as fh: + fh.write(content) +" + +# Append dev banner CSS to stylesheet (only once) +if ! grep -q "dev-banner" "$OUTPUT_DIR/css/style.css" 2>/dev/null; then + cat >> "$OUTPUT_DIR/css/style.css" << 'EOF' /* Dev banner */ .dev-banner { @@ -29,5 +47,6 @@ cat >> "$OUTPUT_DIR/css/style.css" << 'EOF' font-weight: 600; } EOF +fi -echo "Dev banner added successfully!" +echo "Dev banner added to all pages successfully!" diff --git a/scripts/build-docs.sh b/scripts/build-docs.sh index 3f84efbd..8e25b0cf 100755 --- a/scripts/build-docs.sh +++ b/scripts/build-docs.sh @@ -173,6 +173,7 @@ cp site/sdrf-terms.html "$OUTPUT_DIR/" cp site/quickstart.html "$OUTPUT_DIR/" cp site/sdrf-explorer.html "$OUTPUT_DIR/" cp site/sdrf-editor.html "$OUTPUT_DIR/" +cp site/sdrf-builder.html "$OUTPUT_DIR/" # Copy SDRF terms TSV (if present) — also create TERMS.tsv alias for AsciiDoc links cp sdrf-proteomics/metadata-guidelines/sdrf-terms.tsv "$OUTPUT_DIR/" 2>/dev/null || true @@ -190,7 +191,11 @@ cp site/sdrf-data.json "$OUTPUT_DIR/" # Inject navigation headers echo "Adding navigation headers..." -python3 scripts/inject-headers.py "$OUTPUT_DIR" +if [ "$IS_DEV" = true ]; then + python3 scripts/inject-headers.py "$OUTPUT_DIR" --dev +else + python3 scripts/inject-headers.py "$OUTPUT_DIR" +fi # Transform links (SDRF Explorer links and .adoc to .html) echo "Transforming links..." diff --git a/scripts/inject-headers.py b/scripts/inject-headers.py index d84ed4cf..ad00aa2a 100755 --- a/scripts/inject-headers.py +++ b/scripts/inject-headers.py @@ -14,33 +14,59 @@ from pathlib import Path -# Navigation header templates +# Version link placeholder — replaced at build time based on --dev flag +VERSION_LINK_PLACEHOLDER = '{{VERSION_LINK}}' + +# Navigation header templates (use placeholder for version link) HEADERS = { - 'root': '''
SDRF-Proteomics
''', + 'root': f'''
SDRF-Proteomics
''', - 'tools': '''
SDRF-Proteomics
''', + 'tools': f'''
SDRF-Proteomics
''', - 'guidelines': '''
SDRF-Proteomics
''', + 'guidelines': f'''
SDRF-Proteomics
''', - 'templates': '''
SDRF-Proteomics
''', + 'templates': f'''
SDRF-Proteomics
''', - 'sample_guidelines': '''
SDRF-Proteomics
''', + 'sample_guidelines': f'''
SDRF-Proteomics
''', - 'templates_guide': '''
SDRF-Proteomics
''' + 'templates_guide': f'''
SDRF-Proteomics
''' } -def inject_header(filepath: str, header_html: str) -> None: +def inject_header(filepath: str, header_html: str, is_dev: bool = False) -> None: """Inject navigation header into an HTML file.""" with open(filepath, 'r', encoding='utf-8') as f: content = f.read() + # Resolve version link based on dev/stable mode + if is_dev: + version_link = 'Stable Version' + else: + version_link = 'Dev Version' + resolved_header = header_html.replace(VERSION_LINK_PLACEHOLDER, version_link) + # Add has-doc-header class to body content = re.sub(r'', '', content) # Insert header after opening body tag - content = re.sub(r'(]*>)', r'\1\n' + header_html, content) + content = re.sub(r'(]*>)', r'\1\n' + resolved_header, content) + + with open(filepath, 'w', encoding='utf-8') as f: + f.write(content) + + +def rewrite_version_links(filepath: str, is_dev: bool) -> None: + """Rewrite version links in static HTML pages (index.html, quickstart.html, etc.).""" + with open(filepath, 'r', encoding='utf-8') as f: + content = f.read() + + if is_dev: + content = content.replace( + 'Dev Version', + 'Stable Version' + ) + # No change needed for stable — the source files already have /dev/ links with open(filepath, 'w', encoding='utf-8') as f: f.write(content) @@ -48,48 +74,59 @@ def inject_header(filepath: str, header_html: str) -> None: def main(): if len(sys.argv) < 2: - print("Usage: python3 inject-headers.py ") + print("Usage: python3 inject-headers.py [--dev]") sys.exit(1) + is_dev = '--dev' in sys.argv output_dir = Path(sys.argv[1]) + print(f"Dev mode: {is_dev}") + # Inject header into specification.html spec_file = output_dir / "specification.html" if spec_file.exists(): print(f"Injecting header into: {spec_file}") - inject_header(str(spec_file), HEADERS['root']) + inject_header(str(spec_file), HEADERS['root'], is_dev) # Inject header into tools.html tools_file = output_dir / "tools.html" if tools_file.exists(): print(f"Injecting header into: {tools_file}") - inject_header(str(tools_file), HEADERS['tools']) + inject_header(str(tools_file), HEADERS['tools'], is_dev) # Inject header into sample-guidelines.html sg_file = output_dir / "sample-guidelines.html" if sg_file.exists(): print(f"Injecting header into: {sg_file}") - inject_header(str(sg_file), HEADERS['sample_guidelines']) + inject_header(str(sg_file), HEADERS['sample_guidelines'], is_dev) # Inject header into templates.html (templates guide) tpl_guide = output_dir / "templates.html" if tpl_guide.exists(): print(f"Injecting header into: {tpl_guide}") - inject_header(str(tpl_guide), HEADERS['templates_guide']) + inject_header(str(tpl_guide), HEADERS['templates_guide'], is_dev) # Inject headers into metadata-guidelines pages guidelines_dir = output_dir / "metadata-guidelines" if guidelines_dir.exists(): for html_file in guidelines_dir.glob("*.html"): print(f"Injecting header into: {html_file}") - inject_header(str(html_file), HEADERS['guidelines']) + inject_header(str(html_file), HEADERS['guidelines'], is_dev) # Inject headers into template pages templates_dir = output_dir / "templates" if templates_dir.exists(): for html_file in templates_dir.glob("*.html"): print(f"Injecting header into: {html_file}") - inject_header(str(html_file), HEADERS['templates']) + inject_header(str(html_file), HEADERS['templates'], is_dev) + + # Rewrite version links in static HTML pages (index.html, quickstart.html, etc.) + for static_page in ["index.html", "quickstart.html", "sdrf-terms.html", + "sdrf-explorer.html", "sdrf-editor.html", "sdrf-builder.html"]: + static_file = output_dir / static_page + if static_file.exists(): + print(f"Rewriting version links in: {static_file}") + rewrite_version_links(str(static_file), is_dev) print("Header injection complete!") diff --git a/site/index.html b/site/index.html index cede63d4..b8469e7c 100644 --- a/site/index.html +++ b/site/index.html @@ -20,6 +20,7 @@ Tools Explorer Editor + Template Builder Contributors Dev Version GitHub @@ -91,8 +92,13 @@

What does an SDRF file look like?

Quick Start

-

New to SDRF-Proteomics? Start here to learn the basics and create your first SDRF file.

- Get Started +

New to SDRF-Proteomics? Learn the key concepts behind SDRF files and how to fill them in.

+ Learn the Basics +
+
+

Template Builder

+

Build a customized SDRF template by selecting your technology, organism, and experiment type.

+ Build a Template

Validation

diff --git a/site/quickstart.html b/site/quickstart.html index ea6441da..adba83e2 100644 --- a/site/quickstart.html +++ b/site/quickstart.html @@ -20,6 +20,81 @@ line-height: 1.6; margin-bottom: 0.75rem; } + .concept-section { + max-width: 800px; + margin: 0 auto 2.5rem; + } + .concept-section h3 { + margin-bottom: 0.5rem; + color: var(--text-dark); + } + .concept-section p { + line-height: 1.7; + margin-bottom: 0.75rem; + } + .concept-section code { + background: #f4f4f5; + padding: 0.15rem 0.4rem; + border-radius: 4px; + font-size: 0.92em; + } + .concept-columns { + display: grid; + grid-template-columns: 1fr 1fr; + gap: 1.5rem; + margin: 1rem 0; + } + @media (max-width: 640px) { + .concept-columns { + grid-template-columns: 1fr; + } + } + .concept-card { + background: #fff; + border: 1px solid #e5e7eb; + border-radius: 8px; + padding: 1.25rem; + } + .concept-card h4 { + margin-bottom: 0.5rem; + font-size: 1rem; + } + .concept-card p { + font-size: 0.93rem; + margin-bottom: 0.25rem; + } + .concept-card .example-vals { + font-size: 0.85rem; + color: var(--text-light); + } + .rule-list { + padding-left: 1.25rem; + margin-bottom: 1rem; + } + .rule-list li { + margin-bottom: 0.5rem; + line-height: 1.6; + } + .cta-builder-box { + max-width: 800px; + margin: 2rem auto; + background: linear-gradient(135deg, #eff6ff, #f0fdf4); + border: 1px solid #bfdbfe; + border-radius: 12px; + padding: 2rem; + text-align: center; + } + .cta-builder-box h3 { + margin-bottom: 0.5rem; + } + .cta-builder-box p { + color: var(--text-light); + margin-bottom: 1.25rem; + } + .cta-builder-box .btn { + display: inline-block; + font-size: 1.05rem; + } @@ -32,6 +107,7 @@ Tools Explorer Editor + Template Builder Quick Start Dev Version GitHub @@ -41,14 +117,14 @@
- +

What is an SDRF file?

SDRF (Sample and Data Relationship Format) is a tab-delimited file that describes your proteomics samples and links them to data files. Every row is one sample-to-file relationship, and every column captures a piece of - metadata — from the organism you studied to the instrument that acquired the data. + metadata — from the organism you studied to the instrument that acquired the data.

Columns fall into three categories: @@ -95,182 +171,154 @@

What is an SDRF file?

Data / instrument Experimental factors
- -
-

Build your own SDRF below

- ▼ -
- -
- - -
-

1 What type of proteomics experiment?

-

Choose the technology platform used in your study.

-
- -
-
- Why does this matter? ▾ -
-
-

- MS-based and affinity-based proteomics have fundamentally different data types. - MS experiments produce spectra files and need columns for instrument settings, - fragmentation, and search parameters. Affinity experiments (Olink, SomaScan) - produce protein-level measurements and need panel/assay identifiers. - They cannot be combined in a single SDRF. -

-

Read the full specification →

-
-
+ +
+

One row = one data file

+

+ Each row in an SDRF file represents one data file (e.g. a .raw or .mzML file) and the biological sample it came from. + If the same sample was run twice, it gets two rows. If a TMT multiplex packs 10 samples into one file, that file gets 10 rows — one per channel. +

+

+ This design makes the relationship between samples and data explicit, so analysis tools can automatically match files to conditions without manual configuration. +

+
- - + +
+

Three types of columns

+

Every SDRF column belongs to one of three categories. Understanding these makes filling in a template straightforward.

- -
- - + +
+

Factor values tell tools what you're comparing

+

+ Factor value columns are what make an SDRF file useful for analysis. They tell tools like + quantms which sample properties differ between your experimental groups. +

+

+ If you're comparing healthy vs. disease samples, add factor value[disease] + with the same values as characteristics[disease]. If you're comparing + treatment vs. control, add factor value[compound]. + Multiple factor values are allowed when comparing more than one variable. +

+
- -