Phase 1c memo: what course workbenches look like elsewhere
Scope and limits. This is a desk review of vendor and project documentation, retrieved 2026-09-25/26. There was no hands-on use or demo. Feature descriptions are what the documentation claims; marketing pages are treated as claims, not evidence.
I could not tell which curriculum-management system produced the Course Build workbooks. The “MEA Event ID” column didn’t match any product documentation. This is an open question for the curriculum office, because that system may already show course directors some of what follows.
What the systems offer course directors
| System | Type | Course-director-relevant features (per docs) |
|---|---|---|
| Elentra (consortium/cloud) | Commercial; common in US/Canadian medical schools | See below the table |
| one45 (Acuity Insights) | Commercial | See below the table |
| Ilios (UCSF) | Open source (PHP/Symfony + Ember) | See below the table |
| Leo (DaVinci Education) | Commercial LMS + mapping | Real-time curriculum map; gap/redundancy identification; themes/threads mapped to sessions and evaluation questions; learner-facing view of expectations |
| MedHub (UME module) | Commercial | Curriculum mapping, competency tracking, session details for accreditation, scheduling |
Elentra: - Curriculum Explorer: pick a tag (objective, clinical presentation, competency) and see where it was applied (courses, events, gradebook assessments), filtered by course, cohort and year. - Curriculum Matrix: tags × courses inventory. - Curriculum Search: Boolean search over event titles, descriptions and tag names. An advanced option searches the title and description of attached files. - Reports: - Learning Events by Course Missing Objectives; - Course Objectives by Events Tagged (course objectives vs event tags); - Total Hours Taught by Learning Event Type; - untagged assessments; - AAMC Curriculum Inventory export (MedBiquitous). - The course Reports tab is limited to attendance and a teacher list.
one45: - “Gap and redundancy reporting”, assessment coverage, teaching types, program-objective traceability and contact hours. - Maps to multiple frameworks (EPAs, Bloom, themes, keywords). - A Canvas Alignment Tool syncs events and facilitators from Canvas.
Ilios: - Objective hierarchy: session → course → program objectives, linked to competency domains. - Self-service Curriculum Reports across courses and years, with shareable report links and CSV download: - Session Objectives: course, session, session type, objective, instructors, first offering, hours, link; - Session Offerings; - Learner Groups; - Instructional Time: minutes per offering.
Sources: elentra.com/features/curriculum-management-and-mapping; help.elentra.com (Faculty Use of Curriculum Tools & Tags, Course Reports, Curriculum Reports, Learning Event Types); docs.elentra.org (Learning Event Reports); acuityinsights.com/products/one45/curriculum-management; iliosproject.gitbook.io (Curriculum Reports, Competencies and Objectives); davinci-ed.com (Leo); medhub.com (UME).
The common model, and its blind spot
Every system above is built on the same idea. Staff tag events and assessments with objectives and frameworks. Reports count the tags. - Coverage means a tag was applied. - A gap means no tag. - Redundancy means the same tag appears in several places. - Hours are event durations summed by type.
That makes their view exactly as good as the tagging. From the documentation, none of them look inside the teaching material: - Elentra’s file search covers file titles and descriptions, not slide text. - None mention objective-to-content alignment, year-over-year change in materials, or campus-to-campus divergence of materials.
Our corpus is the other half: - 138k pages of slide text and speaker notes; - linked to the same events and objectives the tagging systems use.
Recent literature points the same way, based on abstracts only: - a topic model over all pre-clerkship documents at Yale (PLOS ONE 2023); - UMLS-based analysis of learning-objective terminology (J Med Educ Curric Dev 2026); - a tool that automates lecture-transcript-to-objective coverage; - “Twelve tips to harness the power of AI for curriculum mapping” (Med Teach 2025).
Positioning: a content-evidence layer that sits next to the school’s curriculum-management system. It doesn’t replace it or duplicate its tag-count reports. IDEAS §20 anticipated this.
UI patterns worth borrowing
- Tag explorer: pick a thing, see everywhere it appears (Elentra Curriculum Explorer). We can do this for concepts found in the material, not only tags. Search results already carry course, week and event.
- Matrix/heat grid (Elentra Curriculum Matrix). Weeks × systems or MEPOs for one course, with counts that drill down.
- Completeness reports (Elentra “events missing objectives”). Our equivalents:
- events with objectives but no material;
- Uploaded/Final with no file;
- unscheduled objective titles (DQ1).
- Shareable, parameterized report links and CSV export (Ilios). Course directors forward things to committees. Every table should export.
- Hours by event type (Elentra, Ilios). Learner and scheduled hours (ADR 0003).
- Traceability chains (Ilios objective hierarchy, one45 “program objective traceability”): session objective → MEPO → Foundational Competency/EPA, via our crosswalks.
Patterns to avoid: - Coverage percentages as headline KPIs. They invite reading tag counts as quality (ADR 0005, IDEAS §22). - “Gap/redundancy” labels without evidence. Every flag we show links to the slides behind it.
Recommended Phase 2 feature list (PCV)
Informed by the Phase 1 question set and memos:
| View | Answers | Evidence shown | Borrowed from |
|---|---|---|---|
| Overview | “What does my course look like?” events, learner/scheduled hours by method and week, objectives, Bloom as written (templates split out) | counts drill down to events | Ilios instructional time, Elentra hours by type |
| Schedule map | weeks × sessions with method, hours, objective count, material status | session → files → SharePoint | Elentra event list |
| Objectives | each objective with MEPO → competency/EPA chain, verb and Bloom level, flags (no material / low overlap / template / higher-order with no in-material activity) | top 3 teaching slides per objective | Ilios session-objectives report + our alignment evidence |
| Search | “where is X taught / first introduced / revisited” in this course (and optionally across courses) | slide snippets, ordered by week, duplicates collapsed | Elentra curriculum search, but over content |
| Matrix | weeks × system / chief concern / MEPO | cell → objectives/events | Elentra curriculum matrix |
| Change | what changed vs last year per session (slides added/removed/edited) | slide-level diff | none (new) |
| Campus | AMC vs FCB material per session (shared, adapted, separate) | paired decks | none (new) |
| Data quality | this course’s DQ items | rows | Elentra completeness reports |
Delivery format. Recommendation: a single self-contained HTML file per course, generated by a script like site/index.html: - data as embedded JSON; - client-side search over the course’s slide text (a MiniSearch/Lunr-style index; PCV AY25-26 is about 11k pages); - links to SharePoint for the files themselves.
Why: - a course director can open it from SharePoint or email with no server, login or IT ticket; - access stays with SharePoint; - it’s easy to throw away after feedback.
The alternatives: - A Quarto dashboard: good for Overview and Matrix, weak for interactive search. - A small local app (Streamlit/Shiny/Datasette): needs a host. It’s premature until the pilot shows people want live cross-course queries.
Risk: embedding course text in an HTML file means it goes wherever the file goes. For a pilot handed only to that course’s director, this matches existing access. Note it in the handoff.
Open questions
- Which system does the school use for curriculum management and the AAMC CI/SCOPE submission (“MEA Event ID”)? Does it already give course directors a tag explorer or hours report? If so, link to it rather than rebuild it.
- Can we get an export from that system (events, tags, hours)? It would replace some workbook parsing and fix DQ1 at the source.
- Pilot course director: who, and when?