Books — capture <li> vocab bullets the extractor was silently dropping
extract_epub.py was walking <p> only, but every "Vocabulario" section in the Olly Richards EPUB lives inside <ul><li>...</li></ul>. That meant the heading made it through but the entries didn't — 680 vocab lines across 24 sections in this book were missing from the bundled JSON. Audit (text-node owner by closest block ancestor) confirmed <li> is the only silent drop: 5,260 nodes in <p>, 1,960 in <li>, 0 anywhere else. No <h1>-<h6>, tables, or blockquotes in this EPUB at all. Fix: walk find_all(["p", "li"]) in document order so bullet entries slot in right after their "Vocabulario" / list heading. Re-extracted (2,646 → 3,326 paragraphs), re-translated all 118 jobs in parallel Claude Code subagents. translate_chapters.py prompt template now tells subagents to keep bilingual `palabra = meaning` lines verbatim — both sides already coexist on the line. Bumped bookDataVersion to 2 so refreshBooksDataIfNeeded re-seeds. Verified in simulator: all 13 chapter row sizes grew (e.g. ch6 18,295→20,951 chars). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -141,14 +141,13 @@ def _extract_paragraphs(zf: zipfile.ZipFile, zip_path: str) -> list[str]:
|
||||
return []
|
||||
soup = BeautifulSoup(html, "lxml")
|
||||
paragraphs: list[str] = []
|
||||
for p in soup.find_all("p"):
|
||||
# Drop nav-anchor wrappers that contain no real text.
|
||||
text = _normalise(p.get_text(" ", strip=True))
|
||||
# Walk <p> and <li> in document order so vocab bullets (rendered as
|
||||
# <ul><li>...</li></ul> in this EPUB family) are kept alongside narrative
|
||||
# paragraphs. `<li>` rolls up its inline <b>/<span> children via get_text.
|
||||
for el in soup.find_all(["p", "li"]):
|
||||
text = _normalise(el.get_text(" ", strip=True))
|
||||
if not text:
|
||||
continue
|
||||
# Drop chapter-heading paragraphs that only echo the title — handled
|
||||
# separately by the TOC. Heuristic: very short paragraph that's just
|
||||
# numbers + the chapter title pattern. Keep everything else.
|
||||
paragraphs.append(text)
|
||||
return paragraphs
|
||||
|
||||
|
||||
Reference in New Issue
Block a user