Copy Agent

The Copy Agent is a writing agent that learns from evidence, not a permanent style prompt. It prepares a person's real writing into a voice, chats with that voice, records their preferred edits, builds a new candidate from those edits, evaluates it on held-out cases, and changes the active agent only when the person promotes it. It's a command-line app built only on June's public packages: apps/copy-agent.

Problem

The hard part isn't generating prose. It's preserving authorship, turning corrections into structured evidence, and proving a new version is better without silently changing what answers.

A model with a style prompt writes in its own voice, and it doesn't learn when a person fixes a draft. The Copy Agent works from what they've actually written: it retrieves samples like the request and drafts in their form. When they edit a reply, the edit is kept as evidence, and the next version is graded on cases it never learned from.

Architecture

In thirty seconds:

corpus → prepared voice → chat → edits → evidence split → candidate → held-out eval → promote

In full:

corpus ──prepare──▶ prepared voice (an agent) ──▶ chat ──▶ replies
                                                    │
                          /good  /bad  /edit ◀──────┘
                                    │
                        feedback in the session journal
                                    │
                       evidence: datasets and eval cases
                                    │
                candidate: the active voice plus learnable edits
                                    │
               evaluation: both agents on held-out cases and a benchmark
                                    │
                    promote (a person decides) ──▶ the active agent

A person decides. Feedback, evidence, candidates and evaluations change nothing on their own. Promotion is the only way the active agent changes after the first one, and it refuses without a complete evaluation against the agent it would replace.

• Corpus: a JSONL file or a folder of Markdown and text files, read into a June corpus.
• Agent: a prepared voice, saved under a hash of its content. One agent is active at a time, named in active.json.
• Chat: sessions are June chat journals. Each reply records which agent wrote it.
• Feedback: ratings, notes with #tags, and preferred edits, stored as June feedback on the turn they judge.
• Evidence: datasets and eval cases derived from every session, by rules the app owns.
• Candidate: a new agent, the active voice plus the edits the evidence allows learning from, with a record of its parent.
• Evaluation: both agents run the same workflow on the same cases, scored by deterministic and judged graders, and compared case by case.

Everything is plain JSON under one directory, .copy-agent/ by default.

June modules used

It doesn't use june/trainers. Adding a person's edits to their voice needs no GPU or served model, and a handful of edits is too few to fine-tune on. The conversation dataset it writes is ready for fineTune() when there's enough.

Execution flow

One turn

Each chat turn is one run of <CopyAgent>. The brief step reads the conversation into a copy request with structured output; its result is the argument to <Write>, so the write waits for the brief. The voice then retrieves exemplars like the request, drafts several candidates, scores them against the person's writing, and keeps the best.

<JuneAgent model={model} instructions={briefInstructions(types)}>
  <Step
    id="brief"
    prompt={[...messages, { role: "user", content: input }]}
    output={brief}
  >
    {(fields) => (
      <Write id="draft" voice={voice} request={requestOf(fields)}>
        {children}
      </Write>
    )}
  </Step>
</JuneAgent>

The writer never sees the chat, only the brief and the voice's exemplars. The brief carries facts and requests, never style, so the exemplars decide how the reply sounds.

What this taught June. The voice's model calls first ran outside the runtime, so a turn's trace and usage counted only the brief: 1,217 tokens for a five-call turn. <Write> made the write a step, and the same turns now report 5–7k tokens.

What a real model found in the app. Every scripted test passed while the brief padded a tweet request with hashtags and an emoji, which the writer copied over the person's style. /why showed it, and the brief now carries facts, never style. A real corpus found another: free-form brief types like homepage-hero matched none of the corpus's labels, so every request retrieved the same few posts. The brief now picks from the voice's own kinds of writing.

Chat

The chat answers each turn by rendering <CopyAgent> through agentResponder. Its version is the agent's id, so every reply in the journal says which agent wrote it, even after another is promoted. A child step saves why the reply reads as it does (the request, the exemplars, every candidate's score), which /why prints.

return createChat({
  store: fileStore(join(home, "sessions")),
  responder: agentResponder({
    id: "copy-agent",
    version: agent.id,
    render: ({ sessionId, turnId, messages, input }) => (
      <CopyAgent
        voice={voice}
        types={types}
        model={model}
        messages={messages}
        input={input}
      >
        {(write) => (
          <RecordDraft
            home={home}
            draft={draftOf(write, { sessionId, turnId, agent: agent.id })}
          />
        )}
      </CopyAgent>
    ),
    reply: ({ steps }) => replyOf(steps),
  }),
});

Evidence

copy-agent evidence reads every session. June's converters turn the journals into datasets and eval cases; the app decides which turns count. Edits are split by turn, deterministically, into what a candidate may learn from and what's held out, so nothing is graded on writing it was given.

const edits = preferenceDatasetFromSessions(sessions).filter(kept);
const split = edits.split({ ...SPLIT, key: (e) => turnKey(e.metadata) });
const keys = async (data: typeof edits) =>
  (await data.examples()).map((example) => turnKey(example.metadata));
const learn = await keys(split.train);
const heldOut = await keys(split.test);
const learnable = new Set(learn);

const preferences = split.train;
const conversations = datasetFromSessions(sessions).filter(({ metadata }) =>
  learnable.has(turnKey(metadata)),
);
const cases = evalCasesFromSessions(sessions, {
  negative: true,
  positive: true,
}).filter((c) => kept(c) && !learnable.has(turnKey(c.metadata)));

Approved replies become cases that expect the reply, and rejected ones become cases to beat. A turn tagged #exclude, or one the Copy Agent didn't answer, is left out of everything.

Candidate and evaluation

copy-agent candidate adds the learnable edits to the active voice as new samples, each labelled with the brief of the turn it corrected, and saves a new agent with its parent, its evidence and the turns it learned from. It's never active when it's made. Learning from edits covers the loop this follows.

copy-agent eval runs the current agent and the candidate on the same cases. The held-out edits are the point: the candidate is graded on corrections it never saw. The eval target is the same <CopyAgent> workflow a chat turn runs, so chat and evaluation can't drift apart:

const input = String(messages.at(-1)!.content);
const { steps, usage } = await run(
  <CopyAgent
    voice={voice}
    types={types}
    model={model}
    messages={messages.slice(0, -1)}
    input={input}
  />,
  { signal },
);

There are two suites, never mixed: the held-out cases from chat, and a fixed benchmark of 20 requests across nine writing modes. Both agents are scored by the same graders, with the active agent's voice as the reference, and compareEvaluations() lines the runs up case by case. A judge also picks which reply the person would rather have, asked twice with the replies swapped; if the answers disagree, it's a tie.

What this taught June. Graders can abstain. A grader that doesn't apply to a case returns undefined instead of failing it, so cases with and without an expected reply share one suite.

Promotion

copy-agent promote refuses unless there's an evaluation against the active agent, on the current evidence and benchmark, in which every case was written and scored. Then it shows that evaluation, asks, and records the decision:

const evaluation = await evaluationFor(home, candidate, benchmark);
const why = {
  previous: evaluation.baseline,
  evaluation: `evals/${evaluation.baseline}..${candidate}.json`,
  ...(reason && { reason }),
};
await setActiveAgent(home, candidate, why);

After a promotion the reasons are on disk: active.json names the evaluation, which names the candidate, whose agent.json names its parent, its evidence and the turns whose edits it learned.

Using it

Build June's packages, set a provider key, and run the commands in order from apps/copy-agent:

bun run copy-agent prepare fixtures/wren.jsonl --name Wren
bun run copy-agent chat
bun run copy-agent evidence
bun run copy-agent candidate
bun run copy-agent eval
bun run copy-agent promote <candidate>
bun run copy-agent status

• prepare learns a voice from a corpus. The first agent becomes active.
• chat writes with the active agent. Inside it, /why shows the request, the samples it used and every candidate's score; /good and /bad rate the last reply, with an optional note and #tags; /edit <text> records what you'd have written instead, and /edit alone opens $EDITOR.
• evidence turns every session's feedback into datasets and eval cases.
• candidate makes a new agent from the active voice and the learnable edits.
• eval compares the candidate with the active agent, and prints where the judge preferred the active agent and which cases got worse.
• promote <candidate> makes it active if a person agrees; reject <candidate> --reason "…" records that they didn't.
• status shows the active agent and why, the evidence, and every candidate.

copy-agent help lists every command, including sessions, show and feedback. The chat is also a default export, so June's own CLI runs it: june ask src/chat.ts "…".

Testing

The tests run real June compositions with deterministic models at the edges, so they need no API key:

• In-process tests use AI SDK mock models.
• Tests that run the CLI as its own process use msw to answer the Anthropic and OpenAI HTTP APIs, so the real provider code runs without a network.
• test/loop.test.ts runs the whole loop through the real binary: prepare, chat with edits and ratings, evidence, candidate, eval, promote, then chat again with the promoted agent. It checks that every file on disk agrees with the others.
• test/failures.test.ts checks that a refused or failed step leaves the home directory exactly as it was.

Scripted models can't show whether the writing is any good. A real-model acceptance run (openai:gpt-5-mini writing, openai:gpt-5 judging, six turns, three edits) evaluated a candidate on the evidence and the benchmark. The judge preferred the candidate 9 times, the active agent 8 times and tied 3. A candidate learned from three edits was no clear win, and that's what the evaluation reported.

The benchmark is a JSON file of eval cases, each with a writing mode, a reason it exists and optional deterministic checks. Its graders are deterministic (matches, voice, markers, constraints) and judged (brief, fidelity, authorship, invention), and every judged score keeps the judge's reason. There's no single overall score: the evaluation shows each grader, the preference and the regressed cases for a person to weigh.

What stayed in the app

June provides sessions, feedback, datasets, eval cases, evaluation and comparison. These decisions belong to the Copy Agent:

• The agent manifest and the active pointer. An agent is a prepared voice saved by content hash. active.json says which one answers chats.
• Candidate lineage. A candidate records its parent, the evidence it was made from and the turns whose edits it learned. June has no concept of a candidate.
• The promotion decision. What an evaluation must contain before a person can promote, and the record of every promotion and rejection in decisions.jsonl.
• Corpus ingestion. Reading a folder of Markdown with front matter into June's corpus format.
• Curation rules. Which turns are evidence, #exclude as the way to leave one out, and the learn/hold-out split.

What building it taught June

Building the Copy Agent was also a test of June's public API. OBSERVATIONS.md logs each finding, and AUDIT.md sums them up. Two changed June, and they're next to the parts they changed: the write became a step (One turn), and graders learned to abstain (Candidate and evaluation). The rest were fixes to the app itself, like keeping the brief to facts.

June.dev's own copy is drafted with the Copy Agent, from a voice prepared from 134 samples of its author's writing. A person reviews every piece and records why each draft was approved or edited, and the voice's evaluation says it isn't ready to write unedited yet (apps/www/voice).

Source and related docs

• Source: apps/copy-agent, with its README and benchmark.
• React as the workflow and Agent and Step: the runtime <CopyAgent> runs on.
• Chat, Responders and Feedback and preferred edits: sessions, agentResponder and feedback.
• Copy and voice and Writing with a voice: preparing a voice and <Write>.
• Learning from chat, Evals and Comparing candidates: evidence, evaluation and promotion.
• CLI: june run, june chat and june ask.