Skip to content

Changelog

2026-03-08

  • Performed full Language Engine architecture audit across services/controllers/routes/migrations/docs and mapped existing implementations to canonical module names without creating duplicates.
  • Added canonical additive engine adapter services: lexicalRoleEngineService, semanticConstraintsEngineService, collocationEngineService, sentenceConstructionEngineService, difficultyEngineService, studentLemmaProgressEngineService, placementEngineService, and rubricEngineService.
  • Verified required schema tables (lemma_roles, student_lemma_progress, lemma_frequency, sentence_patterns, conversation_turns, conversation_lemmas) are implemented and connected in service flows.
  • Updated architecture/engine/exercise/word-detail/schema documentation to reflect implemented module mappings, table usage, service-route connectivity, and audit outcomes.

2026-03-08

  • Documentation updated to reflect current Content Bank architecture including taxonomy system, chunk graph, AI classification engine, verb conjugation engine, and vocabulary difficulty scoring.
  • Updated Content Bank architecture docs to describe canonical tables (taxonomy_categories, taxonomy_values, content_item_taxonomies, vocabulary_components, chunk_components, lemma_assets, lemma_grammar, verb_conjugations) and vocabulary schema fields used by the Lexical Unlock Graph.
  • Updated architecture/system docs with the Language Learning Engine component map (Vocabulary Graph, Chunk Graph, Sentence Graph, Taxonomy Classification Engine, Lexical Unlock Graph, Grammar Engine, Exercise Generation Engine, AI Classification Engine, Conversation Engine) and interaction flow.
  • Updated schema snapshot and Word Detail panel docs to reflect taxonomy-based classification, chunk composition, grammar-topic language scope, and asset/grammar/conjugation integrations while preserving backward compatibility.

2026-03-07

  • Language Engine Expansion — Morphology Layer.
  • Added migration sql/0063_morphology_engine.sql with additive morphology tables: lemma_forms, morphology_features, lemma_morphology_features, inflection_paradigms, lemma_paradigm_assignments, and irregular_forms.
  • Added services/morphologyService.js with morphology APIs for lemma-form retrieval, feature-based form resolution, morphology detection, irregular lookup, paradigm lookup, and form-correctness evaluation.
  • Integrated morphology checks into grammar validation and exposed morphology analysis/warnings in grammar-engine responses.
  • Integrated morphology-aware form resolution and fallback behavior into exercise generation flows.
  • Expanded vocabulary detail aggregation payload with morphology.paradigm, morphology.forms, morphology.irregular_forms, and morphology.frequency_by_form.
  • Added WordsManager Morphology UI support (paradigm card, editable forms table, irregular forms, and frequency insights).

2026-03-07

  • Added migration sql/0061_conversation_infrastructure.sql with optional conversational storage tables conversation_turns and conversation_lemmas (+ indexes) for dialogue tracking without replacing existing exercise/sentence models.
  • Added services/conversationEngineService.js and admin endpoint POST /admin/content-bank/conversation/process-turn to process conversational turns via the existing grammar + lexical unlock validation flow.
  • Conversation validation now reuses the lexical unlock rule (introduced_unit_id < current_unit OR introduced_unit_id = current_unit AND introduced_step <= current_step) and returns out-of-scope vocabulary signals.
  • Expanded system documentation for Language Learning Engine architecture (Vocabulary Graph, Chunk Graph, Sentence Lemma Graph, Lexical Unlock Graph, Grammar Validation Engine, Sentence Pattern Engine, Exercise Generation Engine, Conversation Interaction Layer).
  • Expanded engine/module specification docs with detailed purpose/inputs/outputs/interactions for Lexical Unlock Graph, AI Grammar Engine, Sentence Pattern Engine, Exercise Generation Service, and Conversation Engine.
  • Expanded content-bank and schema docs to include conversational lemma usage and graph relationships across sentence_lemmas, chunk_components, and conversation_lemmas.
  • Added docs/engine/EXERCISE_ENGINE.md documenting generation flow, supported exercise types, and conversational-mode reuse.

2026-03-07

  • Added migration sql/0060_ai_grammar_engine.sql introducing sentence_lemmas, chunk_components, and sentence_patterns with indexes and integrity constraints.
  • Upgraded sentence/chunk unlock gating to prefer explicit lexical graph tables (sentence_lemmas, chunk_components) with backward-compatible fallback to sentence_tokens / vocabulary_components.
  • Added services/aiGrammarEngineService.js for structured sentence validation (unlock checks, adjective position detection, correction payloads, follow-up generation).
  • Added services/exerciseGenerationService.js for pattern-based exercise generation constrained to unlocked vocabulary/chunks and active sentence_patterns.
  • Added admin endpoints:
  • POST /admin/content-bank/grammar-engine/validate-sentence
  • POST /admin/content-bank/grammar-engine/generate-followups
  • POST /admin/content-bank/exercises/generate
  • Extended sentence/chunk creation flows to persist sentence_lemmas and chunk_components without breaking existing tables.
  • Updated architecture, engine, word-detail, vocabulary architecture, and schema docs to document grammar engine extensions and safety model.

2026-03-07

  • Content Bank Create Lemma modal accessibility hardening: added dialog description to satisfy Radix DialogContent accessibility requirement.
  • Create Lemma AI action UX improved: auto-classify button now enables as soon as lemma text exists (disabled only while loading) and includes explicit AI iconography.
  • POST /admin/content-bank/lemmas/ai-classify extended to return grammar topic suggestions (grammar_topics + grammar_topic_ids) with taxonomy-only mapping and Unknown fallback behavior.
  • Create Lemma AI prefill now applies grammar topic suggestions into the editable grammar topics multi-select.
  • Safety contract reinforced in docs: AI classification remains assist-only and never auto-persists lemmas.

2026-03-07

  • Lexical Unlock Graph: added migration sql/0059_lexical_unlock_graph.sql introducing vocabulary.introduced_step and progression indexes for unit+step gating.
  • Engine API: course-scoped vocabulary/sentence listing now accepts step-aware progression (student_current_unit, student_current_step, optional student_current_unit_id) and filters future lemmas/chunks/sentences.
  • Content Bank UI: Create Lemma dialog now captures introduced_step to persist unlock position metadata.
  • Taxonomy: added idempotent migration sql/0058_language_scoped_grammar_topics.sql to introduce taxonomy_values.language_id and language-scoped grammar topic seeding (EN/PT/TR/ZH examples).
  • Taxonomy integrity: 0058 now enforces grammar-topic language scoping directly at taxonomy_values level (grammar_topics must carry a valid languages taxonomy reference).
  • Content Bank API: PUT /admin/content-bank/lemmas/:id/classifications now validates grammar-topic language compatibility and returns invalid_grammar_topics_language on mismatch.
  • Taxonomy API/UI support: taxonomy values listing now returns language_id, enabling language-scoped grammar topic selectors in admin UI.
  • Content Bank UI: Create Lemma grammar-topic selector now resolves course context (/admin/courses/:id) and filters topics to course target_language_id.
  • Docs: updated ENGINE_SPEC, ARCHITECTURE, WORD_DETAIL_PANEL_SPEC, and schema snapshot docs to formalize grammar topics as language-scoped taxonomy values.
  • Migration simplification: removed optional link-table backfill/detection logic from 0058; migration now focuses on taxonomy_values.language_id, grammar-topic scope validation trigger, and idempotent language-scoped seed data.

2026-03-04

  • Stabilization: final cleanup pass completed for language-scoped Content Bank migration; course vocabulary listing/creation now resolves courses.target_language_id -> vocabulary.language_id with legacy fallback (vocabulary.course_id) and added debug logs for scope resolution.
  • AI Orchestrator: fortalecido detector de erro de assinatura RPC (run_readonly_sql/run_mutation_sql) para cobrir PGRST202 e variações de mensagem PostgREST.

2026-03-03

  • Segurança SQL do Copilot/Spok: nova migration idempotente sql/0023_spok_sql_guardrails.sql com public.sql_audit_log para auditoria de execuções, incluindo executed_by, rpc_name e sql_text.
  • Adicionadas RPCs public.run_readonly_sql(sql text) (somente SELECT/EXPLAIN/descoberta) e public.run_mutation_sql(sql text) (somente INSERT/UPDATE/DELETE) com bloqueio explícito de DDL/comandos perigosos.
  • Criada allowlist administrável public.sql_mutation_allowlist para restringir mutações às tabelas autorizadas do projeto, sem expor service_role no frontend.
  • Hardening adicional: limite de tamanho de SQL, bloqueio de múltiplos statements e statement_timeout curto (5s) em ambas as RPCs.
  • Permissões: EXECUTE revogado de public e concedido apenas para service_role quando a role existir no ambiente.

2026-03-02

  • Global Linguistic Engine: nova migration idempotente sql/0018_global_linguistic_engine.sql com camada semântica universal (universal_concepts), relações multilíngues (vocabulary_relations, translation_pairs), taxonomia temática (themes, theme_links), integração curricular (curricular_nodes, lesson_content_map) e governança de IA (content_generation_jobs, ai_prompt_versions).
  • Workflow editorial: vocabulary e sentences passam a suportar status e campos de override (ai_general_instructions, image_prompt_override, tts_voice_override, translation_override_instructions).
  • Backend Admin: novo endpoint assíncrono POST /admin/content-bank/batch-generate para criação de jobs de geração em lote com theme, pos, count, target_languages e constraints.
  • Documentação: docs/ENGINE_SPEC.md atualizado com contrato do batch generator e ciclo editorial padronizado.

2026-03-02

  • Content Bank (Grammar): nova migration idempotente sql/0012_grammar_content_bank.sql com grammar_cards, grammar_card_examples, grammar_card_assets e extensão de grammar_concepts (base_lang, level, updated_at).
  • Backend: novas rotas admin para CRUD de conceitos/cards, exemplos/assets, QA/status e busca de sentenças (/admin/content-bank/grammar-*), além de rotas públicas /content-bank/grammar-cards e /content-bank/grammar-cards/:id.
  • UI Admin: novas telas Next.js para Grammar Concepts e Grammar Cards com editor completo, checklist QA e indicação de áudio EN de exemplos via tts_assets.
  • Qualidade: adicionados testes unitários do fluxo de grammar content bank (tests/grammarContentBankController.test.js).

2026-03-02

  • Practice Engine: novo endpoint público POST /practice/evaluate com validação de body (sentence_id, observed, source) e contrato global { ok: true, data } / { ok: false, error } com códigos 400, 404 e 500.
  • Engine Core: implementado services/morphologicalEvaluationService.js com pipeline v1 (normalize NFKC+casefold, tokenização, carregamento de template sentences/sentence_tokens, alinhamento por índice, avaliação lexical/morfológica/sintática e score 40/40/20).
  • Qualidade: adicionados testes em services/morphologicalEvaluationService.test.js cobrindo casos correct, wrong_morphology, wrong_form, wrong_order, extra, missing e consistência de score.

2026-03-02

  • Admin Content Bank: adicionado POST /admin/content-bank/lemmas/:id/forms/generate com modos suggest|save, política overwrite_policy=keep_manual, preview de conflitos e persistência idempotente em lemma_forms sem duplicatas por lemma_id + form + feature_signature (assinatura lógica de features).
  • Backend: novo gerador morfológico EN v1 (verb/noun/adjective) com override prioritário de irregulares via docs/irregulars/en_verbs.json.
  • Admin UI (Word Detail): nova ação Gerar formas com preview editável (form/features), alerta de conflitos e salvamento com recarga de detalhe.
  • Qualidade: testes unitários do gerador para go, study, watch, box e idempotência do cálculo de inserção.

2026-03-02

  • Governança: instituído o Enterprise Documentation Engine com wiki pública em docs.spokeplus.com, estrutura oficial em /docs, snapshot automático de schema e pipeline de deploy contínuo com MkDocs + GitHub Pages.
  • Governança: PRs com mudança estrutural sem atualização dos docs passam a ser inválidos por política.

2026-03-02

  • Docs: criada a especificação oficial do Morphological Evaluation Engine token-level em docs/MORPHOLOGICAL_EVALUATION_SPEC.md com contrato versionado de resultado, camadas (lexical/morphology/syntax), normalização multimodal e roadmap v1.0+.
  • Docs: criada a especificação de geração morfológica docs/MORPHOLOGY_GENERATION_SPEC.md com política lemma-first, regras EN por POS, mecanismo de override irregular e contrato da ação administrativa “Gerar formas”.
  • Docs: criada a especificação de UI/contrato do painel lexical em docs/WORD_DETAIL_PANEL_SPEC.md, incluindo seções obrigatórias e payload do endpoint GET /admin/content-bank/lemmas/:id/detail.
  • Docs: adicionado dataset inicial de irregulares em docs/irregulars/en_verbs.json e links cruzados no ENGINE_SPEC.md, README.md e MANUAL_MESTRE.md.

2026-03-02

  • Governança de snapshot estrutural oficial criada com sql/9000_schema_tables.sql (tabelas) e sql/9001_schema_columns.sql (colunas).
  • Nova migration idempotente 0011_schema_snapshot_view.sql cria/atualiza a view vw_schema_snapshot para inspeção estrutural oficial.
  • Documentação obrigatória atualizada (README + MANUAL_MESTRE) com checklist pós-migration para eliminar divergência entre schema real e docs.

2026-03-02

  • PR: Morphological universal multilingual content bank
  • Nova migration 0010_morphological_engine.sql com lemma_forms, morphology_rules, semantic_relations, novos campos em vocabulary e extensão de sentence_tokens.
  • API admin atualizada com GET /admin/content-bank/lemmas/:id/detail e validação obrigatória de tokens com lemma_id existente ao criar sentenças.
  • Risco/migração: aplicar 0010 após 0009; alteração é idempotente e sem mudança de portas/PM2.

2026-03-02 — Fase E (TTS Enterprise VPS-only)

  • Added idempotent migration sql/0013_tts_assets.sql with definitive tts_assets schema, multi-voice indexes, and status hardening.
  • Added provider-agnostic TTS service layer (services/tts/*) with deterministic hash cache and storage persistence.
  • Added BullMQ queue module and dedicated worker (workers/ttsWorker.js).
  • Added admin enqueue flow for sentence mark-ready and queue monitoring endpoint /admin/system/queues/tts.
  • Updated grammar card ready flow to asynchronous processing_audio before final ready.
  • Added unit tests for queue dedupe and worker status transitions.
  • Updated docs for Redis local setup and PM2 worker.

2026-03-02 — AI Copilot + Documentation Engine Integration (baseline)

Added

  • services/ai-orchestrator/ com provider config, intenção, tools allowlist e PATCH PLAN.
  • Endpoints admin de copilot (/admin/copilot/messages, /admin/copilot/history).
  • Tabela ai_action_logs (sql/0014_ai_action_logs.sql).
  • Endpoint interno de busca documental (GET /internal/docs/search) + índice docs/docs-search-index.json.
  • Rota /admin/docs/metadata com versão e commit hash dos docs.
  • Integração ElevenLabs consolidada na API principal.
  • Copilot lateral inicial no dashboard web e rota /admin/docs.
  • Documento docs/AI_COPILOT_ARCHITECTURE.md.

2026-03-04

  • Added idempotent migration sql/0027_course_content_and_app_cms.sql introducing course_content_items plus App CMS tables (app_projects, app_themes, app_pages, project_course_bindings).
  • Added course-scoped Content Bank endpoints (POST /admin/courses/:id/content-bank/link, GET /admin/courses/:id/content-bank/:type) and advanced filters.
  • Added App CMS admin endpoints (/admin/app-projects*) and public domain bootstrap endpoint (GET /admin/app-projects/domain/:domain).
  • Added word detail actions for manual assets, generate-audio queueing, and explicit image provider-not-configured behavior.
  • Updated admin web UI for course selection gate, scoped filters, App CMS page, and dark-mode-safe token usage.

2026-03-07

  • Lexical Unlock Graph: added migration sql/0059_lexical_unlock_graph.sql introducing vocabulary.introduced_step and progression indexes for unit+step gating.
  • Engine API: course-scoped vocabulary/sentence listing now accepts step-aware progression (student_current_unit, student_current_step, optional student_current_unit_id) and filters future lemmas/chunks/sentences.

  • Content Bank UI: Create Lemma dialog now captures introduced_step to persist unlock position metadata.

  • Vocabulary Content Bank UI redesigned as a course-scoped lexical editor in /courses/[courseId]/content-bank/lemmas with structured lexical list columns, quick filters, and cross-entity search (lemma/translation/sentence text).

  • Vocabulary Content Bank documentation aligned to taxonomy-driven architecture and formalized as the canonical lexical knowledge graph model.
  • Documented official Word Detail lexical editor behavior, including dynamic POS panels and the Save All lexical workflow.
  • Documented chunk builder constraints for multiword expressions composed from existing lemma components.
  • Documented number lemma representation using numeric_value + spelled_form for recognition and transformation exercises.
  • Updated schema snapshot docs to emphasize taxonomy classification (taxonomy_categories, taxonomy_values, content_item_taxonomies) and legacy mirror-field status.
  • Create Lemma flow upgraded to modal with multi-POS taxonomy selection and lemma-level metadata capture (CEFR, frequency, register, semantic domain, lemma type).
  • Word Detail upgraded with tabbed lexical workflow (Core, Morphology, Senses, Translations, Sentences, Pronunciation, Semantic Relations, Taxonomies, Chunks) and single Save All persistence action.
  • Backend compatibility maintained: existing endpoints preserved while adding enhanced list/search metadata and broader classification persistence (status, numeric_value, spelled_form).
  • Added migration sql/0054_vocabulary_taxonomy_ui_support.sql for number dual-representation columns and guaranteed taxonomy seeds for special tokens (symbol POS + token_roles).
  • Documentation updated across architecture/operations/schema to reflect the new vocabulary taxonomy workflow and data model links.

2026-03-07

  • Lexical Unlock Graph: added migration sql/0059_lexical_unlock_graph.sql introducing vocabulary.introduced_step and progression indexes for unit+step gating.
  • Engine API: course-scoped vocabulary/sentence listing now accepts step-aware progression (student_current_unit, student_current_step, optional student_current_unit_id) and filters future lemmas/chunks/sentences.

  • Content Bank UI: Create Lemma dialog now captures introduced_step to persist unlock position metadata.

  • Added migration sql/0056_hierarchical_curriculum_taxonomy.sql with hierarchical taxonomy support (taxonomy_values.parent_id) and self-referential FK/index.

  • Reused existing taxonomy categories (languages, semantic_domains, unit_types, cefr_levels, frequency_bands, registers, parts_of_speech) and added/ensured curriculum_units + grammar_topics without duplicating existing categories.
  • Seeded curriculum progression taxonomy (Unit 1..Unit 300) and linked units to unit_types via parent taxonomy relationships.
  • Extended vocabulary compatibility with introduced_unit_id and updated classification APIs to persist curriculum_units (exactly one introduced unit) and grammar_topics.
  • Updated Create Lemma modal with hierarchical semantic-domain selector, curriculum-unit selector, and grammar-topic multi-select.
  • Added progression gating support in scoped vocabulary queries using student_current_unit (rule: introduced unit must not exceed learner unit).
  • Updated architecture/engine/word detail/schema documentation to reflect hierarchical taxonomies and curriculum-unit integration.
  • Added AI-assisted lemma pre-classification in Create Lemma (Auto classify with AI) with backend endpoint POST /admin/content-bank/lemmas/ai-classify, taxonomy-safe mapping, Unknown fallback, and no automatic persistence.

2026-03-07

  • Lexical Unlock Graph: added migration sql/0059_lexical_unlock_graph.sql introducing vocabulary.introduced_step and progression indexes for unit+step gating.
  • Engine API: course-scoped vocabulary/sentence listing now accepts step-aware progression (student_current_unit, student_current_step, optional student_current_unit_id) and filters future lemmas/chunks/sentences.

  • Content Bank UI: Create Lemma dialog now captures introduced_step to persist unlock position metadata.

  • Added migration sql/0057_vocabulary_difficulty_score.sql to introduce vocabulary.difficulty_score with range check/index and CEFR+frequency backfill for existing rows.

  • Added vocabulary difficulty derivation service with taxonomy/frequency/grammar/abstraction heuristic and AI suggestion parsing.
  • Added admin endpoint POST /admin/content-bank/lemmas/:id/recompute-difficulty supporting recompute strategies (taxonomy, ai).
  • Extended AI classify response to include normalized difficulty_score suggestion.
  • Extended course-scoped adaptive vocabulary filtering with student_difficulty_level and ascending score ordering.
  • Updated Word Detail UI with Difficulty Score field + recompute actions (taxonomy and AI).
  • Updated architecture/engine/schema/word-detail docs for the new adaptive difficulty model.

2026-03-07

  • Added migration sql/0062_language_engine_expansion.sql introducing final additive language engine tables: lemma_roles, lemma_semantic_classes, verb_object_constraints, modifier_constraints, student_lemma_progress, pattern_difficulty, lemma_frequency, and collocation_strength.
  • Extended exercise generation to consume optional role/semantic/collocation/difficulty signals while preserving fallback behavior.
  • Extended grammar validation with optional semantic-constraint and naturalness hints.
  • Extended conversation processing with optional student_lemma_progress updates after validated turns.
  • Extended vocabulary detail aggregation with lexical role, semantic class, frequency detail, collocation insights, and student usage analytics sections.
  • Added additive admin UI enhancements in vocabulary list/tooling visibility (difficulty and introduced step columns + grammar/exercise tooling actions).
  • Added dedicated engine docs: docs/engine/GRAMMAR_ENGINE.md, docs/engine/CONVERSATION_ENGINE.md, docs/engine/SEMANTIC_CONSTRAINTS.md, docs/engine/STUDENT_PROGRESS.md.