# Stage 3: Slide Multimedia Content ## Goal Allow an author to add one or more interactive multimedia items or downloadable resources to any converted PowerPoint slide. Supported content includes uploaded audio, narration, uploaded video, trusted external video embeds such as YouTube, and common document files. Every multimedia item is an independent layer with its own placement, appearance, playback, control visibility, accessibility, and completion settings. Multimedia may be decorative or may be required before the learner can continue. This stage extends the existing PowerPoint-to-SCORM workflow without changing slide rendering. Popup videos, separate video steps, and Stage 2 questions are coupled only through one explicit author-controlled course-flow order; overlay video and audio remain part of their slides. ## Implementation status Stage 3 is implemented across the Laravel API, React authoring interface, full preview, packaged SCORM player, compact resume state, and package builder. Uploaded audio/video normalization requires FFmpeg and FFprobe plus a worker consuming the dedicated `media` queue. Downloadable resources retain their original bytes and do not use the media worker. ## Video presentation modes and unified course flow Every video uses exactly one presentation mode: - **Overlay:** a draggable, resizable layer inside its slide. It is not a course step. - **Popup:** a Small, Medium, or Large modal over the associated dimmed slide. Required unfinished video cannot be dismissed; optional video can expose Close/Skip. - **Separate step:** a responsive video stage after its anchor slide or at course end, with Previous/Next navigation and a video entry in the menu. For each slide and the end-of-course group, the Course flow editor interleaves questions, popup videos, and separate video steps. Every eligible item occurs exactly once. New items are appended to their placement group, missing references are repaired, deleted references are removed, and cross-conversion, duplicate, or placement-mismatched references are rejected. End-of-course video steps occur before assessment review and final submission. ## Storage and API contract Conversion-scoped media documents, items, assets, caption tracks, and course-flow documents use independent optimistic revisions. Media items use UUIDs and stable non-reused numeric tracking keys. Uploaded originals remain private for replacement/reprocessing. Normalized audio/video and validated original downloadable resources are packaged. Token-protected authoring endpoints are: - `GET|PUT /api/v1/powerpoint-to-scorm/conversions/{id}/media` - `POST /api/v1/powerpoint-to-scorm/conversions/{id}/media/uploads` - `POST /api/v1/powerpoint-to-scorm/conversions/{id}/media/{media}/replacement` - `POST /api/v1/powerpoint-to-scorm/conversions/{id}/media/{media}/captions` - `POST /api/v1/powerpoint-to-scorm/conversions/{id}/media/{media}/retry` - `GET|PUT /api/v1/powerpoint-to-scorm/conversions/{id}/course-flow` Processed media, posters, and captions use temporary signed routes. Complete media and course-flow saves are atomic, reject stale revisions with `409`, and invalidate an existing package. Generate first saves dirty course settings, course flow, questions, and media. ## Finalized limits and processing - Video: 200 MB each; audio: 50 MB each; downloadable resource: 50 MB each; WebVTT: 2 MB each. - Total source media: 500 MB per conversion; media items: 200 per conversion. - Video output: MP4, H.264, AAC 128 kbps, `yuv420p`, fast-start, aspect preserved, maximum 1080p, plus JPEG poster. - Audio output: MP3 at 128 kbps. - Accepted source families: MP4, MOV, M4V, WebM, MKV, AVI, MP3, M4A, AAC, WAV, and OGG, subject to FFprobe inspection. - Downloadable resources: PDF, DOC/DOCX, XLS/XLSX, PPT/PPTX, TXT, and ZIP, validated by extension and file content. Package generation is rejected while any referenced uploaded asset is queued, processing, failed, missing, or lacks a usable output. Stable package paths use `media/{tracking-key}/media.mp4`, `audio.mp3`, `poster.jpg`, caption filenames, and `resources/{tracking-key}/{safe-original-name}`. ## Course completion and resume Canonical course completion supports `slides_only`, `slides_and_required_media`, `slides_and_required_questions`, and `slides_media_and_required_questions`. The prior quiz completion value remains a compatibility alias, while the package and new frontend use the course setting. The backward-compatible `v3` SCORM state decoder continues to accept v1 slide state and v2 question state. V3 stores slide progress, assessment state, stable media keys, completion, unique 1% watched buckets, play-once state, technical waivers, and resumable positions. Authoring rejects a worst-case state above the portable SCORM 1.2 4,096-character `suspend_data` ceiling. Required media or resources gate Next and forward menu navigation while preserving backward navigation. A resource completes when its download link is activated; this does not mark its slide viewed and does not claim the operating system finished saving the file. Timer auto-advance remains pending until the last gate clears. A required YouTube failure exposes retry and then a technical waiver; the learner may continue, but the item is explicitly shown as unavailable rather than watched. ## Authoring flow 1. Upload and process the PowerPoint presentation. 2. Configure the slide player and, when needed, questions. 3. Open the separate **Media** category. 4. Select a slide. 5. Add one or more audio, narration, uploaded-video, embedded-video, or downloadable-file items. 6. Position and resize visual media over the rendered slide. 7. Configure playback, visibility, controls, captions, and completion behavior for each item. 8. Configure how multiple media items interact on the slide. 9. Preview the complete learner experience. 10. Generate the SCORM 1.2 package. ## Supported multimedia The initial implementation supports: - Uploaded video files. - Uploaded audio and narration files. - YouTube videos added using a validated video URL. - WebVTT caption or subtitle files for uploaded video. - An optional accessible transcript for audio and video. - Downloadable PDF, Word, Excel, PowerPoint, text, and ZIP resources. Each downloadable resource is attached to one slide and uses one presentation mode: a draggable, resizable, fully styled button inside the slide, or the application-styled Resources control between Previous and Next. One player-area resource downloads directly; multiple resources open a responsive accessible menu. Buttons support icon and text, icon only, or text only, including an optional custom icon. Resource completion is independent of slide viewing and can optionally be required. A slide can contain multiple items of the same or different types. For example, one slide may contain two narration tracks, an uploaded video, a YouTube video, and several short audio explanations. Multimedia is rendered as interactive HTML layers over the converted slide image. It is not inserted back into the original PowerPoint file and does not reconstruct PowerPoint multimedia timelines. ## Media layers Each multimedia item has its own stable identifier and can be: - Positioned over the slide using drag-and-drop. - Resized while preserving its aspect ratio by default. - Reordered above or below other multimedia layers. - Duplicated. - Replaced without losing its placement and settings where compatible. - Temporarily hidden in the authoring interface. - Deleted. The editor should provide an ordered list of all media on the selected slide so an author can select an item even when layers overlap or the item is hidden. Audio items may use a visible player, a compact control, a custom icon, or no persistent visual layer. Video items use a visible rectangular layer whose dimensions and position are stored relative to the original slide size so the layout remains responsive. ## Playback triggers Each media item can start: - Manually when the learner selects its Play control. - Automatically when the slide opens. - After a configured delay from the time the slide becomes active. - After another media item on the same slide finishes. - When the learner selects a configured button or hotspot. A dependency on another media item must reference an item on the same slide. Circular playback dependencies are invalid and must be rejected during authoring and package generation. Each item can also be configured to: - Pause when the learner leaves the slide. - Resume from its previous position when the learner returns. - Restart whenever the slide is revisited. - Play only once after it has been completed. - Loop until the learner stops it or leaves the slide. - Pause other media when it starts. - Play simultaneously with other media when the slide permits it. ## Coordinating multiple media items Each slide has a media playback policy: ### Exclusive playback Starting one audio or video item pauses any other playing media on that slide. This is the recommended default and prevents narration, video audio, and sound effects from overlapping unintentionally. ### Simultaneous playback Multiple audio and video items may play at the same time. Authors can use playback triggers and delays to create intentional sequences or synchronized content. A per-item setting may override the slide default when simultaneous playback is intentional for a specific item. ## Audio and narration display Every audio or narration item supports one display mode: - Full audio player. - Compact audio player. - Play and Pause button only. - Custom audio icon. - Hidden narration with no persistent player. When a visible or custom player is used, the author can show or hide the following controls where supported: - Play and Pause. - Seek bar. - Current playback time. - Total duration. - Volume and Mute. - Playback speed. - Restart. - Download. Hidden narration may continue playing while its controls are hidden. If the narration is optional, the author may choose whether a temporary Play, Pause, or Mute control appears while it is active. A required media item must never become impossible to start or complete. If autoplay is blocked or fails, the player must expose an accessible fallback action even when the configured display mode is hidden. ## Video display and controls Uploaded and embedded video items support: - Draggable and resizable placement. - A poster image before playback where supported. - Full controls, minimal controls, or a custom control surface. - Play and Pause. - Seek bar. - Current time and duration. - Volume and Mute. - Full-screen mode. - Captions. - Playback speed where supported. - Restart. - Download for uploaded video when explicitly enabled. The exact controls available for an external provider depend on that provider's player API and policies. The preview must accurately show when a requested control cannot be hidden or customized for an embedded video. ## Visibility settings Every media item supports a visibility rule: - Always visible while its slide is active. - Hidden until playback begins. - Visible only while playing. - Hidden after completion. - Visible after a configured delay. - Visible after another media item on the same slide completes. Playback and visibility are separate. For example, an audio narration can remain hidden while it plays, and a completed video can remain visible on its final frame. An item that is hidden must not remain keyboard-focusable. When it becomes visible, focus should not move automatically unless the learner initiated the action or focus movement is required to prevent a navigation trap. ## Autoplay behavior Browsers commonly block audible autoplay until the learner interacts with the page. Therefore: - Autoplay is treated as a requested behavior, not a guarantee. - Playback may begin automatically after a learner-initiated navigation action such as selecting Next. - If first-slide narration is blocked, the course displays a clear **Start course** or **Play narration** action. - Muted video may autoplay where the browser permits it. - Autoplay failure must not leave a required media item or the Next button permanently locked. - Preview must demonstrate the same fallback behavior as the generated course. ## Captions, subtitles, and transcripts Uploaded video supports one or more WebVTT caption tracks. Each track includes: - Language. - Human-readable label. - Caption or subtitle kind. - Optional default-track selection. Audio and video may also include a text transcript. The learner can open and close the transcript using an accessible control. The transcript must be available to keyboard and screen-reader users. Caption and transcript requirements: - Uploaded caption files must be validated before saving. - Caption language and label are required. - Caption controls remain keyboard accessible. - Text contrast and focus states follow the existing DocuDeck player design. - Captions and transcripts are included in the generated SCORM package for uploaded media. Automatic transcription and automatic caption generation are not included in the initial Stage 3 implementation. ## Media completion rules Every media item uses one completion rule: - Does not affect progress. - Complete when playback starts. - Complete after the learner watches or listens to a configured percentage. - Complete when playback reaches the end. For percentage-based completion: - The configured percentage must be between 1 and 100. - Seeking forward must not automatically count skipped content as watched or listened to. - Completion is based on the unique portions of the media actually played. - Replaying the same portion does not increase the completed percentage. An item may be marked required or optional. If a slide contains multiple required media items, every required item must satisfy its own completion rule before media-based navigation requirements are satisfied. The locked Next button remains visible and explains what remains, for example: > Watch at least 80% of the video to continue Previously completed media does not have to be replayed unless the author explicitly configures it to restart for presentation purposes. Restarting playback does not remove its recorded completion. ## Relationship to slide progress and questions Slide, media, and question progress remain separate: - Slide progress measures unique required slides viewed according to the Stage 1 rule. - Media progress measures required multimedia items completed. - Question progress and scoring continue to follow Stage 2 rules. The course completion configuration can require: - Slides only. - Slides plus required media. - Slides plus required questions. - Slides plus required media and required questions. Completing media does not answer a question. Answering a question does not complete required media. Reaching a later slide must never complete media that the learner did not actually play according to its configured rule. ## Resume and SCORM 1.2 tracking The generated package stores compact multimedia state in `cmi.suspend_data`, including where practical: - Stable media identifier. - Completion state. - Unique watched or listened ranges needed for percentage completion. - Last playback position for unfinished media. - Whether a play-once item has already played. - Media sequence state required to restore the current slide. The package should: - Save after important playback, completion, navigation, and exit events. - Restore unfinished required media on resume. - Restore the exact media sequence for the active slide. - Avoid automatically starting multiple audible items immediately after resume. - Keep multimedia data compact because SCORM 1.2 has limited portable `suspend_data` capacity. - Preserve existing Stage 1 progress and Stage 2 assessment data. Multimedia is tracked internally because SCORM 1.2 does not define a dedicated standard video or audio progress model. Media completion contributes to the course status only through the configured DocuDeck completion rules. ## Uploaded media processing For consistent playback across supported LMS environments, the backend should use FFmpeg to inspect and normalize uploaded media. Recommended package formats: - Video: MP4 containing H.264 video and AAC audio. - Audio: MP3 or AAC. - Captions: WebVTT. Processing should: - Validate the real MIME type rather than trusting the filename. - Reject corrupt or unsupported media. - Read duration, dimensions, codecs, bitrate, and file size. - Normalize media that is not already in the supported package format. - Generate a poster image for uploaded video. - Preserve the original aspect ratio. - Report processing progress and actionable errors. - Prevent uploaded filenames from becoming unsafe package paths. The application should configure upload-size and duration limits and show them before selection. Exact limits may be deployment-specific. ## Uploaded and externally embedded media ### Uploaded media - Is stored privately during authoring. - Is copied into the generated SCORM ZIP. - Can work without an internet connection after the package is installed. - Provides the most reliable playback and completion tracking. - Increases processing time, storage use, ZIP size, LMS upload time, and learner bandwidth. Before generation, the application displays the estimated package size and warns when the package may exceed common LMS upload limits. ### YouTube media - Is added using a validated YouTube URL or video identifier. - Is rendered using a generated, restricted embed configuration. - Requires an internet connection at learner runtime. - Uses the provider player API for supported playback and completion events. - Is not downloaded or copied into the SCORM package. - May be unavailable when an LMS, browser policy, organizational firewall, or content-security policy blocks the provider. The authoring interface and generated course must show a useful unavailable-media state. Completion settings must explain that externally hosted media is less reliable than uploaded media. Authors must not paste arbitrary iframe HTML. The backend stores a normalized provider and media identifier, and the package generates the iframe safely. Vimeo and other trusted providers can be added later through provider-specific adapters. Direct remote video and audio URLs are deferred until cross-origin access, expiring URLs, hotlink protection, privacy, and tracking behavior are explicitly designed. ## Media settings category Multimedia controls remain separate from slide display, navigation, timers, animations, and questions. The Media category includes: - A slide selector. - An ordered layer list for the selected slide. - Add Audio, Add Video, and Add YouTube actions. - Replace, duplicate, reorder, hide, and delete actions. - Position and size controls for visual layers. - Playback trigger and dependency controls. - Visibility controls. - Player-control visibility settings. - Caption and transcript settings. - Completion and required-status settings. - Slide-level exclusive or simultaneous playback behavior. Essential settings appear first. Advanced sequencing, visibility, and individual control settings remain under collapsed **More options** sections. ## Preview The author can preview the selected slide and the complete course without an LMS connection. Preview includes: - Actual uploaded media and external embeds. - Responsive layer positioning. - Multiple items playing according to the slide policy. - Manual, automatic, delayed, chained, and hotspot playback triggers. - Visible, compact, custom, and hidden audio controls. - Video control visibility. - Captions and transcripts. - Required-media navigation locks. - Percentage and end-of-media completion. - Autoplay-blocked fallback behavior. - Pause, resume, revisit, and play-once behavior. - Interactions with Stage 1 timers and Stage 2 questions. Preview state does not write anything to an LMS and does not permanently mark authoring media as completed. ## SCORM package output The generated SCORM 1.2 ZIP includes: - Uploaded and normalized media files. - Video poster images. - Caption files and transcripts. - Media layout and behavior configuration. - Custom accessible media controls. - Trusted external-provider integration code. - Compact resume and completion tracking logic. - All existing slide, player, and assessment assets. - Validated downloadable resources at stable offline-relative paths. Every packaged local asset must be declared or reachable through the SCORM package resources. Generated asset names must be stable, safe, and collision-free. Package generation must fail with a clear error if a required uploaded asset is missing or still processing. Failure to load optional media at learner runtime must not crash the slide player. ## Security and privacy - Do not accept arbitrary iframe HTML, scripts, or embed markup. - Allow only explicitly supported external providers. - Normalize and validate external media identifiers on the backend. - Generate iframe permissions and sandbox behavior from trusted application code. - Validate uploaded content by MIME type and media inspection. - Store source media on the private disk. - Do not expose author upload paths to learners. - Do not execute metadata, scripts, or active content from uploaded files. - Clearly inform authors that embedded providers receive learner network requests and may set cookies according to their own policies. ## Accessibility - All visible media controls must work with a keyboard. - Controls require accessible names, focus states, and sufficient contrast. - Hidden media must not leave hidden focusable elements. - Required media must always have an accessible way to start, pause, and complete it. - Captions and transcripts must be available without using a mouse. - Custom controls must expose the current state and value to assistive technology. - The player respects reduced-motion preferences for media-layer entrance effects. - Media must not unexpectedly autoplay audible content when doing so would violate browser or accessibility expectations. ## Validation rules - Every media item must belong to a valid slide. - Every item must have a supported media type and a stable identifier. - Uploaded media must finish processing before package generation. - Visual layer position and dimensions must remain within valid slide bounds. - A delayed trigger must use a non-negative duration. - A chained trigger must reference another item on the same slide. - Playback dependencies must not contain cycles. - Percentage completion must be between 1 and 100. - A hidden required item must have a reliable trigger and an accessible fallback control. - Caption files must be valid WebVTT and include a language and label. - YouTube URLs must resolve to a valid normalized video identifier. - Unsupported providers and arbitrary iframe markup must be rejected. - A media item cannot be required when its configuration provides no reliable method of measuring the selected completion rule. ## Acceptance criteria - Authors can add multiple audio, narration, uploaded-video, and YouTube items to the same slide. - Every item has independent layout, playback, visibility, controls, and completion settings. - Authors can drag, resize, reorder, duplicate, replace, hide, and delete media layers. - Audio can use a full player, compact player, button, custom icon, or hidden narration mode. - Authors can control which audio and video controls are presented where the media technology supports it. - Media can start manually, on slide entry, after a delay, after another item, or from a configured learner action. - Exclusive playback prevents accidental audio overlap by default. - Simultaneous playback supports intentional multimedia sequences. - Required media keeps Next disabled until its configured completion rule is satisfied. - Seeking does not count skipped media segments as watched or listened to. - Completed media remains completed after leaving, reopening, and resuming the SCORM course. - Autoplay failure exposes a usable fallback and never creates a permanent navigation lock. - Uploaded media, captions, and transcripts are included in the generated ZIP. - YouTube embeds use validated identifiers rather than author-provided iframe code. - Preview accurately represents multimedia behavior without an LMS connection. - Existing slide progress, questions, scoring, attempts, and completion behavior continue to work. - The generated package works with the Moodle-first SCORM 1.2 target. ## Recommended defaults - Slide playback policy: Exclusive playback. - Audio display: Compact player. - Video controls: Shown. - Autoplay: Disabled. - Loop: Disabled. - Pause when leaving the slide: Enabled. - Resume when returning: Enabled. - Restart on revisit: Disabled. - Media affects completion: Disabled. - Uploaded video download: Disabled. - Captions: Shown when the learner enables them. - External provider: YouTube only in the initial implementation. ## Not included in Stage 3 - Question authoring or changes to question scoring. - Arbitrary iframe or script embedding. - Native PowerPoint multimedia extraction or timeline reconstruction. - Full video or audio editing. - Screen, camera, or microphone recording. - Automatic transcription or caption generation. - Text-to-speech narration generation. - Automatic translation or dubbing. - Live streaming. - Digital-rights-managed media. - Downloading YouTube or other externally hosted videos. - Direct arbitrary remote media URLs. - Branching based on media playback. - SCORM 2004-specific tracking. These capabilities can be discussed in later stages after the core multimedia layer, packaging, accessibility, and tracking behavior are stable.