Skip to content

tutorial.md

Latest commit

 

History

History
246 lines (175 loc) · 6.28 KB

tutorial.md

File metadata and controls

246 lines (175 loc) · 6.28 KB

End-to-End Tutorial: Running Your First Research Session

This tutorial walks through the current Agora Lab workflow:

  • research-staff participates in regular group meetings as an internal scientific judge
  • paper-reviewer appears only in the separate paper-review loop after the supervisor believes the work is close to submission

You will set up a lab with one supervisor, two students, one research staff member, and one paper reviewer, then inspect the files Agora creates at each stage.

Prerequisites

git clone https://github.com/LiXin97/agora-lab.git
cd agora-lab

Step 1: Initialize the Lab

The simplest path is the unified CLI:

./agora init "Efficient attention mechanisms for long-context LLMs" \
  --students 2 \
  --staff 1 \
  --paper-reviewers 1

This creates a .agora/ runtime in your project with:

  • .agora/lab.yaml and .agora/LAB.md
  • generated agents: supervisor, student-a, student-b, staff-a, paper-reviewer-1
  • .agora/shared/KANBAN.md seeded with the supervisor's first task
  • symlinks into the installed runtime (scripts/, hooks/, templates/, skills/)

If you want to call the underlying script directly:

AGORA_PROJECT_DIR="$PWD/.agora" bash scripts/lab-init.sh \
  --topic "Efficient attention mechanisms for long-context LLMs" \
  --students 2 \
  --staff 1 \
  --paper-reviewers 1

Step 2: Add Agents Manually (Optional)

If you initialized without counts, add agents explicitly:

./agora add student-a student -direction "Linear attention variants"
./agora add student-b student -direction "Sparse attention and local-global hybrids"
./agora add staff-a research-staff
./agora add paper-reviewer-1 paper-reviewer

Check the roster:

./agora list

The important distinction is role semantics:

  • research-staff joins regular meetings
  • paper-reviewer stays outside meetings and only enters paper-review cases

Step 3: Start the Lab

Launch all agents and the watchdog:

./agora start
./agora status

Useful runtime commands:

./agora list
./agora watch
./agora attach student-a

At this point the paper reviewer is usually idle. The normal loop begins with student work, supervisor coordination, and group meetings.

Step 4: Understand the Shared State

The key files and directories inside .agora/ are:

.agora/
├── lab.yaml
├── LAB.md
├── agents/
│   ├── supervisor/
│   ├── student-a/
│   ├── student-b/
│   ├── staff-a/
│   └── paper-reviewer-1/
└── shared/
    ├── KANBAN.md
    ├── artifacts/
    ├── meetings/
    ├── paper-reviews/
    └── messages/

Use this mental model:

  1. shared/messages/ — supervisor assignments and decisions
  2. shared/artifacts/ — student outputs and research-staff judgments
  3. shared/meetings/ — regular 5-phase meeting records
  4. shared/paper-reviews/ — submission-readiness review rounds

Step 5: Run a Regular Research Meeting

Use the wrapper:

./agora meeting

Or call the meeting engine directly:

bash scripts/lab-meeting.sh -caller supervisor -new

Regular meetings now include:

  • supervisor
  • students
  • research staff

Regular meetings explicitly exclude paper reviewers.

Meeting outputs live under .agora/shared/meetings/M001/ and follow this structure:

.agora/shared/meetings/M001/
├── agenda.md
├── perspectives/
│   ├── student-a.md
│   └── student-b.md
├── judgments/
│   └── staff-a.md
├── critiques/
├── responses/
└── decision.md

The important workflow change is that meeting-time internal evaluation is stored in judgments/, not reviews/.

Step 6: Open a Paper Review Case

When the supervisor thinks a draft is approaching submission quality, start a paper-review case:

./agora paper-review -new draft-long-context-v1 student-a "paper-reviewer-1"
# Example output:
# Created paper review case P001 for paper 'draft-long-context-v1'
./agora paper-review -status

Here draft-long-context-v1 is your paper ID (chosen by you), while P001 is the case ID generated by Agora. Use the generated P001 / P002 / ... identifiers with -round and -complete-round.

This creates:

.agora/shared/paper-reviews/P001/
├── meta.yaml
├── packet.md
└── rounds/
    └── R1/
        ├── packet.md
        ├── reviews/
        │   └── paper-reviewer-1.md
        └── supervisor-resolution.md

The paper-review loop is separate from meetings:

  1. paper reviewers write round reviews in reviews/
  2. the supervisor writes supervisor-resolution.md
  3. if the resolution says Outcome: ready-for-next-round, open another round:
./agora paper-review -round P001
  1. if the resolution says Outcome: submission-ready, close the active round:
./agora paper-review -complete-round P001

Step 7: Compare Against the Shipped Example Snapshot

If you want a concrete reference without running a full session, inspect the curated example in examples/:

  • examples/shared/meetings/M001/ — a regular meeting with research-staff judgment
  • examples/shared/paper-reviews/P001/ — a separate paper-review case with a reviewer packet and supervisor resolution

Start with:

less examples/README.md

Then inspect:

less examples/shared/meetings/M001/agenda.md
less examples/shared/meetings/M001/judgments/research-staff-1.md
less examples/shared/paper-reviews/P001/meta.yaml
less examples/shared/paper-reviews/P001/rounds/R1/reviews/paper-reviewer-1.md

What to Remember

  • research-staff is the internal judge inside normal meetings
  • paper-reviewer is a separate top-level role used only in paper-review cases
  • group meetings and paper reviews are intentionally separated
  • examples/ is the fastest way to understand the expected file layout and outputs