From RL Video Tutorials to Production RL Engineering Skills
The people building Terminal-Bench and Harbor have shared a lot of hard-won RL engineering knowledge - how they package tasks, how they catch agents cheating, how they turn rollouts into training data. Most of it lives in long conference talks.
Three of those talks add up to about an hour and a half of video. They're worth every minute - and hard to reuse. The next time you build an environment, you're back to scrubbing timelines for the one slide you remember.
So we handed the watching to a team of agents. With Gemini 4 in Eigent, one brief turned three Terminal-Bench talks into four file-based skills for production RL environment engineering, plus a source-linked report that shows where every idea came from.
The Full Workflow at a Glance
You choose the talks. One brief takes it from there to skills your agents can reuse. Every stage leaves a file behind, so you can check the work at each step.
| Stage | What happens | What you get |
|---|---|---|
| Pick | Choose the YouTube talks you want to learn from | A short list of video links in the brief |
| Watch | Download transcripts and frames, and watch each talk in full | A timestamped outline of every talk |
| Analyse | Pull out methods, insights, reported results, and warnings | Learning points labelled OBSERVED, INFERRED, or UPDATED |
| Compare | Line the talks up against each other | Where they agree, where they differ, and how each fills the others' gaps |
| Distill | Group methods and insights by practical capability | A SKILL.md and an evidence.md per skill |
| Validate | Check structure, run scripts, and try a skill on a new task | Each skill marked STRUCTURALLY CHECKED, EXECUTED, or UNTESTED |
| Report | Build and serve an HTML report on 127.0.0.1 | Embedded talks, clickable timestamps, and a coverage matrix that shows gaps |
| Reuse | Point a coworker at a SKILL.md | Production RL environment engineering on your own project |
Here's how to do it yourself, step by step - including one lesson we learned about writing the brief.
What You'll End Up With
- Reusable skills - each a
SKILL.mdwith anevidence.md, written so another agent can apply it without watching the talks. - A source-linked HTML report - served locally, with every talk embedded, a timestamped outline, and learning points you can click to jump straight to the moment.
- A coverage matrix - which skill covers which part of the RL workflow, and which areas none of the talks address.
- An honest paper trail - every claim labelled OBSERVED, INFERRED, or UPDATED, and every skill marked as executed, structurally checked, or untested.
Before You Start
- Eigent installed on your machine.
- A Gemini API key.
- The YouTube links you want your agents to learn from. We used these three:
| Talk | Channel | Length |
|---|---|---|
| Introducing Terminal-Bench: Evaluating LLM Agents in Realistic Terminal Settings | Anyscale (Ray Summit 2025) | 31:23 |
| Everything Is a Rollout - Alex Shaw + Ryan Marten | AI Engineer | 21:11 |
| Terminal-Bench 2.0: Launch + Q&A with the founders | Latent Space | 35:00 |
Add Your Gemini API Key
- Open Settings → Models.
- Find Gemini and open its settings.
- Paste your API key.
- Set the model type to
gemini-4. - Click Save.
Select Gemini 4
- On the home screen, open the model picker under the input box.
- Choose Gemini → gemini-4.
- Set the thinking effort if you want - the default works.
Paste the Brief
Copy the brief below into the Cowork with input box. Swap in your own video links, and set the Docker line to allowed or not allowed.
Watch the YouTube videos listed below and turn their methods and insights into reusable,
file-based agent skills. Then publish the results as a well-designed HTML report served from a
local web server.
Videos:
- https://www.youtube.com/watch?v=zt-U4lDdenY&t=1559s
- https://www.youtube.com/watch?v=jRCpXUjz4CI
- https://www.youtube.com/watch?v=5wo0mLlG0fk&t=1815s
Use only these videos as primary sources. Do not search for, substitute, or add other videos.
You may consult official documentation and the code repositories the videos link to or name,
but only to check or update details shown in the videos.
## Focus
Extract practical, repeatable methods. Where the videos cover them, include:
- Defining tasks, observations, actions, and termination conditions.
- Creating reproducible environments and reset behaviour.
- Building verifiers and reward functions.
- Detecting reward hacking and invalid successes.
- Collecting, scoring, and reviewing agent trajectories.
- Preparing data for agent training and evaluation.
Do not invent coverage for areas the videos don't address. List those areas as gaps instead.
Capture insights, not only procedures. For every method, record why the speakers use it, what
went wrong before they did, any results or numbers they reported, and the warnings they gave.
Treat named tools and frameworks (for example Harbor or Terminal-Bench) as one implementation
of a method. Record the underlying principle first and the tool-specific steps second.
## Permissions and limits
- Allowed: reading the video pages; downloading subtitles, transcripts, and the video files
for analysis; extracting frames; reading official docs and repositories named in the
videos; creating a local virtual environment and installing open-source packages into it;
running a local static web server bound to 127.0.0.1.
- Docker: [allowed / not allowed].
- Not allowed: model training, GPU jobs, paid hosted services, calls to paid LLM APIs, or
publishing anything to a public host. For trajectory collection tests, use a scripted,
rule-based, or random policy.
- Keep all files inside ./rl-env-skills/ in the current working directory. Delete downloaded
video files when finished.
- Do not ask for confirmation for the actions allowed above. Stop and report only if a required
action falls outside them.
- If a video is private, removed, region-locked, or has no usable audio or transcript, say so in
the report. Continue with the remaining videos, and do not replace it.
## Capability honesty
Before starting, record which relevant capabilities you have: video viewing, frame
extraction, transcript access, code execution, browser preview. If you cannot inspect video
visuals, work from transcripts and mark every visual claim as "not observed".
Never describe on-screen content you did not actually see.
## Analysis
For each video:
- Record title, channel, published date, duration, and view count with the date checked.
- Watch it in full. Inspect visual demonstrations, code, diagrams, and narration where your
capabilities allow.
- Build a timestamped outline of the main sections.
- Preserve timestamps for every important procedure, decision, and warning.
- List the key insights and takeaways: design decisions and the reasons behind them, lessons
learned, failure stories, reported results, and warnings. Give each one a timestamp.
- Label each claim OBSERVED (seen or heard in the video), INFERRED (your interpretation), or
UPDATED (adapted to current official docs; cite the doc URL and the date checked).
- Re-inspect unclear moments. Record unresolved details instead of guessing.
- Paraphrase. Quotes must be under 15 words and carry a timestamp. Do not reproduce transcripts.
Then note where the videos agree, where they differ, and where one fills a gap in another.
## Skills
Create 2–4 skills grouped by practical capability. A skill may draw from several videos.
Layout:
rl-env-skills/
<skill-name>/
SKILL.md # required
evidence.md # timestamps, OBSERVED/INFERRED/UPDATED labels, source per point
scripts/ templates/ # only if they materially help; scripts must be tested
report/
index.html # the HTML report (see below)
assets/ # extracted frames or screenshots used in the report, if any
REPORT.md
manifest.json
SKILL.md must begin with YAML frontmatter:
---
name: <kebab-case, matches folder name>
description: <1–2 sentences: what it does and when to use it, including trigger phrases>
---
The body must cover: prerequisites, required inputs, key insights (the principles and lessons
behind the procedure, each with its source and timestamp), procedure (numbered steps), expected
outputs, decision rules, failure recovery, observable success checks, and a tool-agnostic path
for projects that don't use the tools shown in the videos. Another agent should be able to
apply it to a new project without watching the videos.
A skill must not be only a wrapper around one tool's features. If you removed every
tool-specific command, the insights and decision rules should still stand on their own.
Keep these four things distinct throughout: task datasets, recorded trajectories, supervised
training examples, and RL policy optimisation. Never imply that exporting trajectories is RL
training.
## Validation
- Structural: frontmatter is present and valid, the name matches the folder, every referenced
file exists, every evidence timestamp falls within the source video's duration, and every
skill has a key insights section with at least three sourced insights.
- Execution: run every script you created. Where feasible, apply at least one skill end to end on
a small example the videos don't use (e.g. a text grid-world, or a "make this failing unit
test pass" task).
- Report each item as STRUCTURALLY CHECKED, EXECUTED (with command and result), or UNTESTED
(with reason).
## HTML report (report/index.html)
### Content, in this order
1. Header: report title, date checked, the agent's capability summary (had / lacked).
2. Summary: 3–5 sentences on what was produced and the main limitations. No marketing language.
3. Sources: one row per video (title, channel, published date, duration, views with date
checked, linked code or repository, access status).
4. One section per video:
- The original video embedded with a privacy-enhanced YouTube embed
(https://www.youtube-nocookie.com/embed/<VIDEO_ID>), with a 16:9 responsive frame.
- Link to the original.
- Timestamped outline of the video.
- Key insights and takeaways, each with a clickable timestamp and a label.
- Learning points: numbered, each with a clickable timestamp that seeks the embedded player
(YouTube IFrame Player API) and an OBSERVED / INFERRED / UPDATED label.
- Unresolved details.
- If embedding is blocked, show the video thumbnail linked to youtube.com/watch?v=<ID>
and state that embedding was unavailable.
5. How the videos compare: agreements, differences, and how each fills the others' gaps.
6. One section per skill:
- Name, absolute path to SKILL.md, one-sentence purpose, and when to use it.
- Key insights the skill carries, with their sources.
- Procedure summary (the key steps, not the whole file).
- Evidence: every supporting source point as a row — video, timestamp (clickable), label,
and what the skill took from it. Include doc URLs for UPDATED items.
- Validation status with the command and result if executed.
7. Coverage matrix: skills × the videos, showing which video supports which skill, plus
the focus areas no video covered.
8. Data distinctions: a short table defining task dataset, trajectory, supervised example, and
RL policy optimisation, and which of them the skills actually produce.
9. Limitations and untested items.
10. Three copy-paste reuse requests, each in a code block with a copy button.
### Visual design: light mode, in Google's design language
- Style only: do not use Google logos or wordmarks, or suggest the report is a Google product.
- Colours: page #FFFFFF; secondary surfaces #F8F9FA; borders #DADCE0; primary text #202124;
secondary text #5F6368; links and primary actions #1A73E8. Use the Google accent set
(blue #4285F4, red #EA4335, yellow #FBBC04, green #34A853) only for status and labels,
e.g. OBSERVED blue, INFERRED yellow tint with dark text, UPDATED green, UNTESTED red.
Text must meet WCAG AA contrast.
- Type: Google Sans / Google Sans Text if available from Google Fonts, otherwise Roboto; Roboto
Mono or Google Sans Code for code and timestamps. Clear scale: one H1, H2 per section, 16px
body, 1.5 line height, comfortable line length (about 70–80 characters).
- Layout: 8px spacing grid, max content width about 1100px, sticky left table of contents on
desktop that collapses on mobile, flat cards with 1px borders or a subtle Material elevation
shadow, 8–12px corner radius, plain tables with hairline row dividers.
- Avoid the generic "AI-generated" look: no gradients, purple/indigo themes, glassmorphism,
glowing effects, emoji used as icons, sparkle icons, oversized hero banners, decorative
blobs, or hype copy ("unlock", "supercharge", "seamless"). Prefer real content, tables, and
data over decoration.
- Technical: one self-contained HTML file with inline CSS and JS. Only external requests allowed:
Google Fonts and YouTube embed/API. Responsive down to 360px width with no horizontal
page scroll. Semantic HTML, keyboard-accessible controls, alt text on images.
### Serve and verify
1. Start a static server from rl-env-skills/report/ bound to 127.0.0.1, e.g.
`python3 -m http.server 8000 --bind 127.0.0.1`. If port 8000 is busy, use the next free port.
Keep the server running when you finish.
2. Do not rely on opening the file from disk: YouTube embeds often fail without an http origin.
3. Verify: the page returns HTTP 200; every embed loads or shows the fallback; timestamp links
seek the player; all anchors and file links resolve; no console errors. If you have a browser
tool, take a desktop and a 375px-wide screenshot and save them to report/assets/.
4. Report how you verified each item, and which checks you could not run.
## Deliverables
- REPORT.md: a plain-text version of the report's findings.
- manifest.json:
{ "date_checked": "...",
"videos": [{"url","title","channel","published","duration","views","access_status"}],
"skills": [{"name","path","sources":["<video id>", "..."],
"insights": <number of sourced insights>,
"validation":"executed|structural|untested"}],
"capabilities": {"video": bool, "frames": bool, "transcripts": bool, "code_exec": bool,
"browser_preview": bool},
"report": {"path": "<absolute path to index.html>", "preview_url": "http://127.0.0.1:<port>/",
"server_command": "...", "verified": [...], "not_verified": [...]} }
- Final reply: the preview link on its own line first
(http://127.0.0.1:<port>/), then the command to restart the server if it stops, then a short
summary of the videos, skills with absolute paths, validation results, and limitations.
A few parts of this brief do the heavy lifting:
- Use only these videos keeps your agents on the sources you chose - no swaps, no extras.
- Capture insights, not only procedures makes sure the why travels with the how. More on this below.
- Capability honesty stops anyone from describing a video frame nobody actually saw.
- OBSERVED / INFERRED / UPDATED labels keep facts, interpretation, and doc-checked updates apart.
- Data distinctions keep task datasets, trajectories, supervised examples, and RL policy optimisation separate - so exporting trajectories never gets mistaken for training.
What We Changed in the Brief - and Why
The run in our video used an earlier version of this brief. It did a lot right. Gemini 4 watched all three talks, cross-checked details against the Harbor docs and repository, and built four skills. Every skill script ran end to end on a task none of the talks use - repairing a Python config parser called fix-ini-interpolator.
But when we read the skills, we noticed something. They leaned hard on Harbor's features - task folder layouts, trajectory formats, and CLI commands. The insights behind them mostly stayed in the report.
And the talks are full of insights. The report itself captured these:
- Scaffolds matter as much as models. Cancelling stuck commands early was worth about 5–10%, and mixing models in one run was worth about 5–6% (Ray Summit talk, 18:05–19:23).
- Plan for scale early. A laptop tops out around six concurrent containers, which is why the teams moved to cloud sandboxes (all three talks).
- Reward hacking is RL's overfitting. Treat a suspiciously high score the way you'd treat a suspiciously low validation loss (AI Engineer talk, 06:35).
A skill that only knows how to run a tool can't tell you when the approach breaks, or what to do on a stack without that tool. So we made four changes:
- Insights are a first-class output. Every video section and every skill now records the speakers' reasoning, lessons, results, and warnings - each with a timestamp.
- Principle first, tool second. Harbor and Terminal-Bench count as one way to apply a method, not the method itself.
- No tool wrappers. Every skill needs a key insights section and a tool-agnostic path. Strip out the commands, and the skill should still teach something.
- Validation checks for it. Each skill must carry at least three sourced insights, or it fails the structural check.
We also fixed a small mismatch. The earlier brief said "the two videos" but listed three links. Gemini 4 caught it, noted it in the report, and analysed all three - but the new wording works for any number of videos.
Let Your Agents Work
Send the brief. Gemini 4 plans the work and spins up the agents. From there, the run follows the brief:
- Watch each talk - transcript, frames, slides, and live demos - and keep the timestamps.
- Distill the methods and insights into skills, each with its own evidence file.
- Validate the skills and record what was checked, executed, or left untested.
- Generate the report and serve it on
127.0.0.1.
In our run, the agents downloaded all three talks, extracted 25+ keyframes, pulled full timestamped transcripts, and cross-checked Harbor details against the official docs - then cleaned up the downloaded video files, as the brief asks.
Open the Report
The report opens on a local server - in our run, http://127.0.0.1:8000. Here's what you'll find:
- Capabilities - what the agents had and lacked, from video viewing to browser preview.
- Summary and sources - each talk with its channel, publish date, duration, views, and linked repositories.
- One section per talk - the video embedded with a privacy-enhanced player, a timestamped outline, and labelled learning points.
- How the talks compare - where they agree, where they differ, and how each fills the others' gaps.
- One section per skill - purpose, key steps, evidence, and the exact validation command.
- Limitations - what stayed untested and why, like cloud sandboxes and paid model APIs.
Read the Coverage Matrix
The matrix shows which skill covers which part of the RL workflow - and where the talks leave gaps.
| Focus area | Task Scaffolder | Verifier & Anti-Cheat | Rollout Collector | SFT Dataset Preparer |
|---|---|---|---|---|
| Defining tasks, observations, actions & termination | Primary | Secondary | Primary | - |
| Reproducible environments & reset | Primary | Secondary | Secondary | - |
| Building verifiers & reward functions | Secondary | Primary | - | - |
| Detecting reward hacking & invalid successes | - | Primary | Secondary | - |
| Collecting, scoring & reviewing trajectories | - | - | Primary | Secondary |
| Preparing data for SFT training & evaluation | - | - | - | Primary |
| Gap: RL policy-gradient math & trainer internals | Not covered | Not covered | Not covered | Kept separate |
| Gap: Step-level dense process reward shaping | Not covered | Outcome reward only | Not covered | Not covered |
The two gap rows matter. The talks name trainers like SkyRL and Tinker but never show the training loop - so the skills don't pretend to cover it.
Put the Skills to Work
Each skill lives in its own folder inside rl-env-skills/:
sandboxed-task-environment-scaffolder- scaffold and validate reproducible, containerized tasks.verifier-and-anti-cheat-auditor- build outcome verifiers and audit them for reward hacking.rollout-trajectory-collector-and-debugger- run sandboxed rollouts, record trajectories, and classify failures.trajectory-to-sft-dataset-preparer- filter verified trajectories into clean SFT data, kept strictly apart from RL training.
Point a coworker at one and give it a real job, for example:
Use rl-env-skills/verifier-and-anti-cheat-auditor/SKILL.md to design a verifier
for my "make this failing unit test pass" task. Flag any way an agent could
pass without fixing the bug.
The skill carries the procedure, the decision rules, and the success checks. The evidence file carries the receipts - and with the new brief, the insights come along too.
Make It Your Own
The brief works for any set of technical talks. Swap the video links and the focus list, and keep the rest:
- Keep the insight rules. They're what turn a tool manual into hard-won judgment.
- Keep the honesty rules. Capability checks and claim labels are what make the output trustworthy.
- Keep the validation step. A skill that was never run is a note, not a skill.
An hour and a half of talks doesn't have to stay an hour and a half of talks. Give them to your agents - and keep what they learn.
Credits: built with Gemini 4 and Eigent. Talks by Mike Merrill (Ray Summit 2025), Alex Shaw and Ryan Marten (AI Engineer), and the Terminal-Bench founders on Latent Space - watch the originals on YouTube. Music: "Wallpaper" Kevin MacLeod (incompetech.com), licensed under Creative Commons: By Attribution 4.0.
