feat(cms): U3 derive article dates from content/ front-matter
Replace the hand-maintained ARTICLE_DATES registry as the source of
truth with dates read from content/ front-matter, so publishing a post
via the CMS needs no manual registry edit (plan R4).
- tools/content_index.py: load_dates() reads content/ front-matter into
{source_folder/slug: {published|modified: iso}}. Standalone (os+yaml),
no import cycle with seo_inject.
- tools/seo_inject.py: ARTICLE_DATES is now {**static_fallback,
**content_dates}; content wins. The static table is kept only as a
fallback for entries not present in content/.
- migration now preserves the published-vs-modified distinction
(date_type), so the switch is byte-identical: content-derived dates
equal the old static table exactly (17/17, incl. the one 'modified').
Verified: full `seo_inject --all --sitemap` + build_listings +
build_botlinks run produces ZERO changes to any site page. build_content
--check still reproduces all 51 pages.
build_botlinks.py needs no change (it scrapes built pages, which are a
pure function of content/). Listings get content-driven dates for free
via seo_inject.ARTICLE_DATES.
tools/tests/test_content_index.py: 5 tests (dates == legacy registry,
merged registry unchanged, published/modified distinction, events key on
projects/<slug>, one ISO date per entry).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
596290acf8
commit
8ff968739a
|
|
@ -7,6 +7,7 @@ title: How to open a Slovenian bank account as an international student
|
|||
subtitle: 'A practical guide for Macedonian students: the documents you need, how to choose a bank, using Flik, and keeping your details up to date after you open the account.'
|
||||
category: Student life
|
||||
date: '2026-08-01'
|
||||
date_type: modified
|
||||
date_line: Student life · Last checked 1 August 2026
|
||||
hero:
|
||||
src: /images/blog-slovenian-bank-account.webp
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ title: Како да отворите словенечка банкарска с
|
|||
subtitle: 'Практичен водич за македонските студенти: документите што ви требаат, како да изберете банка, користење на Flik и ажурирање на вашите податоци откако ќе ја отворите сметката.'
|
||||
category: Студентски живот
|
||||
date: '2026-08-01'
|
||||
date_type: modified
|
||||
date_line: Студентски живот · Последно проверено 1 август 2026
|
||||
hero:
|
||||
src: /images/blog-slovenian-bank-account.webp
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ title: Kako odpreti slovenski bančni račun kot mednarodni študent
|
|||
subtitle: 'Praktičen vodnik za makedonske študente: dokumenti, ki jih potrebujete, kako izbrati banko, uporaba Flik in posodabljanje podatkov po odprtju računa.'
|
||||
category: Študentsko življenje
|
||||
date: '2026-08-01'
|
||||
date_type: modified
|
||||
date_line: Študentsko življenje · Zadnjič preverjeno 1. avgust 2026
|
||||
hero:
|
||||
src: /images/blog-slovenian-bank-account.webp
|
||||
|
|
|
|||
|
|
@ -61,7 +61,8 @@ body_format: html # html | markdown (see below)
|
|||
| `title` | yes | Plain text; rendered into `<h1>` and `<title>`. |
|
||||
| `subtitle` | no | Blog/event lead paragraph (`.article-subtitle`). News has none. |
|
||||
| `category` | no | Shown before the date on blog pages. |
|
||||
| `date` | yes | ISO date, authoritative for listing order. Sourced from `tools/seo_inject.py` `ARTICLE_DATES`, else extracted from the page. |
|
||||
| `date` | yes | ISO date, authoritative for listing order and for the article's schema date. Since U3 this is the source of truth (`tools/content_index.py` builds the date registry from it; the old `ARTICLE_DATES` table in `seo_inject.py` is now only a fallback). |
|
||||
| `date_type` | no | `published` (default, omitted) or `modified`. Selects `datePublished` vs `dateModified` in the Article schema. |
|
||||
| `date_line` | no | The exact human-readable date line from the page, preserved for fidelity. U2 may instead regenerate it per-language from `date`. |
|
||||
| `hero.*` | no | Lead image + caption. `src` is root-relative (`/images/…`). `caption_html` is verbatim (may contain attribution links). |
|
||||
| `body_format` | yes | See below. |
|
||||
|
|
|
|||
|
|
@ -1,5 +1,19 @@
|
|||
# tools/
|
||||
|
||||
## content_index.py
|
||||
|
||||
Reads the `content/` source tree (front-matter) as structured data. Its
|
||||
`load_dates()` builds the article date registry consumed by `seo_inject.py`,
|
||||
so **adding a post no longer needs a manual `ARTICLE_DATES` edit** — the date
|
||||
lives in the post's front-matter (`date` / `date_type`). See
|
||||
`docs/content-model.md`.
|
||||
|
||||
## build_content.py
|
||||
|
||||
Renders the `content/` source files into the blog/news/event detail pages
|
||||
(`--check` verifies the output matches the current site; `--write` emits pages).
|
||||
See `docs/content-model.md` and the no-code CMS plan in `docs/plans/`.
|
||||
|
||||
## seo_inject.py
|
||||
|
||||
Injects a technical-SEO block into every language page (`en|mk|si/**/index.html`)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,65 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
content_index.py -- read the content/ source tree (U1) as structured data.
|
||||
|
||||
Implements the date half of U3: the hand-maintained ARTICLE_DATES registry in
|
||||
seo_inject.py is replaced by dates derived from content/ front-matter, so adding
|
||||
a post via the CMS needs no manual registry edit. Standalone (only os + yaml) so
|
||||
seo_inject can import it without a cycle.
|
||||
|
||||
See docs/content-model.md for the front-matter schema.
|
||||
"""
|
||||
import os
|
||||
|
||||
import yaml
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
CONTENT = os.path.join(ROOT, "content")
|
||||
LANGS = ("en", "mk", "si")
|
||||
_TYPE_DIRS = ("blog", "news", "events")
|
||||
|
||||
|
||||
def _read_front_matter(path):
|
||||
with open(path, encoding="utf-8") as f:
|
||||
text = f.read()
|
||||
if not text.startswith("---\n"):
|
||||
return None
|
||||
_, front, _body = text.split("---\n", 2)
|
||||
return yaml.safe_load(front) or {}
|
||||
|
||||
|
||||
def _canonical_meta(slug_dir):
|
||||
"""Front-matter for an entry, from en.md if present else any language.
|
||||
Dates/source_folder are language-independent, so any file will do."""
|
||||
for lang in LANGS:
|
||||
p = os.path.join(slug_dir, f"{lang}.md")
|
||||
if os.path.exists(p):
|
||||
meta = _read_front_matter(p)
|
||||
if meta:
|
||||
return meta
|
||||
return None
|
||||
|
||||
|
||||
def load_dates():
|
||||
"""Return {"<source_folder>/<slug>": {"published"|"modified": "YYYY-MM-DD"}}
|
||||
built from content/ front-matter. Empty dict if content/ is absent."""
|
||||
dates = {}
|
||||
if not os.path.isdir(CONTENT):
|
||||
return dates
|
||||
for tdir in _TYPE_DIRS:
|
||||
base = os.path.join(CONTENT, tdir)
|
||||
if not os.path.isdir(base):
|
||||
continue
|
||||
for slug in sorted(os.listdir(base)):
|
||||
meta = _canonical_meta(os.path.join(base, slug))
|
||||
if not meta or not meta.get("date"):
|
||||
continue
|
||||
key = f"{meta.get('source_folder', tdir)}/{slug}"
|
||||
kind = meta.get("date_type", "published") # published is the default
|
||||
dates[key] = {kind: str(meta["date"])}
|
||||
return dates
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import json
|
||||
print(json.dumps(load_dates(), indent=2, ensure_ascii=False))
|
||||
|
|
@ -72,19 +72,23 @@ def _text(node):
|
|||
|
||||
|
||||
def _iso_date(source_folder, slug, en_html=None):
|
||||
"""ISO date: prefer the central ARTICLE_DATES registry, then fall back to
|
||||
reading it off the English page (mirrors build_listings' behaviour)."""
|
||||
"""(iso, kind) for an entry: prefer the central ARTICLE_DATES registry
|
||||
(preserving whether the date is a 'published' or 'modified' one), then fall
|
||||
back to reading it off the English page (mirrors build_listings)."""
|
||||
key = f"{source_folder}/{slug}"
|
||||
rec = seo_inject.ARTICLE_DATES.get(key, {})
|
||||
iso = rec.get("published") or rec.get("modified")
|
||||
if iso:
|
||||
return iso
|
||||
if rec.get("published"):
|
||||
return rec["published"], "published"
|
||||
if rec.get("modified"):
|
||||
return rec["modified"], "modified"
|
||||
if en_html is not None:
|
||||
try:
|
||||
return seo_inject.extract_publish_date(en_html)
|
||||
iso = seo_inject.extract_publish_date(en_html)
|
||||
if iso:
|
||||
return iso, "published"
|
||||
except Exception:
|
||||
return None
|
||||
return None
|
||||
pass
|
||||
return None, None
|
||||
|
||||
|
||||
def _parse_hero(figure):
|
||||
|
|
@ -154,10 +158,13 @@ def parse_page(html, source_folder, slug, lang, en_html=None):
|
|||
if cat and not re.search(r"\bpublished\b|\d{4}", cat, re.I):
|
||||
record["category"] = cat
|
||||
|
||||
iso = _iso_date(source_folder, slug, en_html=en_html)
|
||||
iso, kind = _iso_date(source_folder, slug, en_html=en_html)
|
||||
if not iso:
|
||||
return None, "no date (missing from ARTICLE_DATES and unparseable)"
|
||||
record["date"] = iso
|
||||
if kind == "modified":
|
||||
# 'published' is the implicit default; only record the exception.
|
||||
record["date_type"] = "modified"
|
||||
if date_line:
|
||||
record["date_line"] = date_line
|
||||
|
||||
|
|
@ -175,7 +182,7 @@ def parse_page(html, source_folder, slug, lang, en_html=None):
|
|||
# Deterministic front-matter key order for readable, stable output.
|
||||
_KEY_ORDER = [
|
||||
"type", "slug", "lang", "source_folder",
|
||||
"title", "subtitle", "category", "date", "date_line", "hero", "body_format",
|
||||
"title", "subtitle", "category", "date", "date_type", "date_line", "hero", "body_format",
|
||||
]
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -18,6 +18,8 @@ except Exception:
|
|||
|
||||
# Repo root = parent of this tools/ directory (portable; no hardcoded path).
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
import content_index # noqa: E402 (dates from content/ front-matter, U3)
|
||||
DOMAIN = "https://msosorg.com"
|
||||
SITE_NAME = {"en": "Macedonian Student Organisation in Slovenia",
|
||||
"mk": "Македонска студентска организација во Словенија",
|
||||
|
|
@ -39,9 +41,10 @@ SECTION_NAMES = {
|
|||
}
|
||||
HOME_CRUMB = {"en": "Home", "mk": "Дома", "si": "Domov"}
|
||||
|
||||
# Article dates (ISO), curated once from the English pages (dates are
|
||||
# language-independent). Emitted as datePublished/dateModified in Article schema.
|
||||
ARTICLE_DATES = {
|
||||
# Article dates (ISO). Emitted as datePublished/dateModified in Article schema.
|
||||
# Source of truth is now content/ front-matter (see content_index / U3); this
|
||||
# static table is only a fallback for any entry not present in content/.
|
||||
_STATIC_ARTICLE_DATES = {
|
||||
"blog/how-to-open-slovenian-bank-account": {"modified": "2026-08-01"},
|
||||
"blog/where-to-search-for-accommodation-in-slovenia": {"published": "2026-08-03"},
|
||||
"blog/learning-slovene-where-to-start": {"published": "2026-07-28"},
|
||||
|
|
@ -61,6 +64,13 @@ ARTICLE_DATES = {
|
|||
"projects/watching-handball-together-in-slovenia": {"published": "2025-01-25"},
|
||||
}
|
||||
|
||||
# Content front-matter is authoritative; it overrides the static fallback above.
|
||||
# A new post published via the CMS therefore needs no manual date entry here.
|
||||
try:
|
||||
ARTICLE_DATES = {**_STATIC_ARTICLE_DATES, **content_index.load_dates()}
|
||||
except Exception:
|
||||
ARTICLE_DATES = dict(_STATIC_ARTICLE_DATES)
|
||||
|
||||
# OPTIONAL location overrides for Event schema on /projects/ pages.
|
||||
# By default a project's city is auto-detected from its English page body
|
||||
# (see detect_city), so NEW projects need NO entry here — just publish them.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,60 @@
|
|||
#!/usr/bin/env python3
|
||||
"""
|
||||
Tests for tools/content_index.py + the seo_inject date wiring (U3).
|
||||
|
||||
Core guarantee: deriving ARTICLE_DATES from content/ front-matter reproduces the
|
||||
previously hand-maintained registry exactly (dates AND the published/modified
|
||||
distinction), so JSON-LD / meta output does not regress — while new posts no
|
||||
longer need a manual registry edit.
|
||||
|
||||
Run directly: python tools/tests/test_content_index.py
|
||||
Or with pytest: pytest tools/tests/test_content_index.py
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
TOOLS = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, TOOLS)
|
||||
|
||||
import content_index # noqa: E402
|
||||
import seo_inject # noqa: E402
|
||||
|
||||
|
||||
class DatesMatchLegacyRegistry(unittest.TestCase):
|
||||
def test_content_dates_equal_static_fallback(self):
|
||||
"""content/ front-matter reproduces the old hand-maintained table 1:1."""
|
||||
self.assertEqual(content_index.load_dates(), seo_inject._STATIC_ARTICLE_DATES)
|
||||
|
||||
def test_effective_registry_matches(self):
|
||||
"""The merged ARTICLE_DATES seo_inject actually uses is unchanged."""
|
||||
self.assertEqual(seo_inject.ARTICLE_DATES, seo_inject._STATIC_ARTICLE_DATES)
|
||||
|
||||
def test_published_modified_distinction_preserved(self):
|
||||
dates = content_index.load_dates()
|
||||
# This blog carried a 'modified' date, not 'published' — must survive.
|
||||
self.assertEqual(dates["blog/how-to-open-slovenian-bank-account"],
|
||||
{"modified": "2026-08-01"})
|
||||
# A normal entry defaults to 'published'.
|
||||
self.assertEqual(dates["news/proof-of-means-updated-2026"],
|
||||
{"published": "2026-08-01"})
|
||||
|
||||
|
||||
class RegistryShape(unittest.TestCase):
|
||||
def test_keys_use_source_folder(self):
|
||||
# Events live under content/events/ but must key on projects/<slug>
|
||||
# (their URL + schema key), not events/<slug>.
|
||||
dates = content_index.load_dates()
|
||||
self.assertIn("projects/paint-and-wine-at-sunset", dates)
|
||||
self.assertNotIn("events/paint-and-wine-at-sunset", dates)
|
||||
|
||||
def test_every_entry_has_one_iso_date(self):
|
||||
for key, rec in content_index.load_dates().items():
|
||||
self.assertEqual(len(rec), 1, f"{key}: expected one date kind")
|
||||
(kind, iso), = rec.items()
|
||||
self.assertIn(kind, ("published", "modified"))
|
||||
self.assertRegex(iso, r"^\d{4}-\d{2}-\d{2}$", f"{key}: bad ISO date")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main(verbosity=2)
|
||||
Loading…
Reference in New Issue