Voice preparation

prepareVoice() turns a corpus into a voice's working material: the samples worth learning from, each with a reconstructed prompt and labels, as a dataset with deterministic splits. Most of it is deterministic bookkeeping, but one step calls a teacher model once per sample, and what that model writes is generated metadata you should review before relying on it.

validate ─▶ exclude ─▶ dedupe ─▶ reconstruct each brief + infer labels ─▶ split
(on load)   (rules)    (text)    (one teacher call per kept sample)       (by id)

1. Validation happens when the corpus loads, before preparation starts.
2. Exclusion drops samples include rejects. Deterministic.
3. Deduplication drops near-duplicates of an earlier sample. Deterministic.
4. Teaching: one teacher call per remaining sample reconstructs its brief and labels what its metadata leaves out. This is the only step that calls a model, and the only one whose output differs between runs.
5. Splitting pairs each prompt with the untouched text, and splits by sample id. Deterministic.

Preparation never changes a sample's text, and it's needed for two things: retrieval on a corpus that isn't fully labelled, and fine-tuning. A corpus whose samples already have good labels can write without it.

import { openai } from "june/sdk/openai";
import { copyCorpus, prepareVoice } from "june/copy";

const prepared = await prepareVoice(copyCorpus("./fonzie.jsonl"), {
  teacher: openai("gpt-5"),
});

prepared.samples; // kept samples, each with its prompt and inferred labels
prepared.removed; // [{ id, reason: "excluded" | "duplicate", of? }]
prepared.dataset; // every kept sample as a completion example
prepared.splits; // { train, validation, test }
prepared.corpus; // { source, size, fingerprint } of the corpus it came from

Its options:

Exclusion

By default prepareVoice() keeps every sample except those marked authored: false. Pass include to decide yourself, for example to also leave out writing someone else edited, or writing from before a cut-off:

import { openai } from "june/sdk/openai";
import { copyCorpus, prepareVoice } from "june/copy";

const prepared = await prepareVoice(copyCorpus("./fonzie.jsonl"), {
  teacher: openai("gpt-5"),
  include: (sample) =>
    sample.metadata.authored !== false &&
    sample.metadata.edited !== true &&
    (sample.metadata.date ?? "") >= "2022",
});

include replaces the default rule, so keep the authored check if you still want it. Each excluded sample is listed in removed with reason: "excluded". See authorship hygiene for what to exclude.

Deduplication

After exclusion, prepareVoice() drops near-duplicates, keeping the first of each group in corpus order. A sample is a duplicate of an earlier one when:

• their text matches once case, punctuation and spacing are ignored; or
• their lengths are within the threshold of each other, and their sets of word trigrams overlap by at least the threshold (shared trigrams over all distinct trigrams of both).

The threshold is 0.9 by default; dedupe: { threshold: 0.8 } catches looser copies, and dedupe: false keeps everything. It's deliberately simple and transparent: it catches cross-posted and lightly edited copies, not paraphrases. Each one is in removed with reason: "duplicate" and of, the id of the sample it duplicates.

Reverse instructions

An archive of writing has the answers but not the questions: nobody saved the brief behind each homepage. Training and evaluation need instruction → response pairs, so by default the teacher reconstructs the brief: the instruction an editor or client most plausibly gave to get exactly this text, with its purpose, audience, format and length but none of its wording.

Instruction (synthetic, from the teacher):
  Write a two-line homepage hero for founders that says Fonzie handles sourcing.

Response (authentic, never changed):
  Recruiting, minus the recruiting.
  Fonzie does the sourcing so you can do the building.

The distinction matters. The completion is always the original text, byte for byte. The teacher only writes prompts and labels; it's never asked to improve or rewrite anything, and nothing it says ends up in a completion. The teacher sees each sample's text and its string metadata, so it can use what's known (its type, its audience) when writing the brief.

To write prompts yourself instead, pass a function. It can be async, and must return a non-empty string:

import { openai } from "june/sdk/openai";
import { copyCorpus, prepareVoice } from "june/copy";

await prepareVoice(copyCorpus("./fonzie.jsonl"), {
  teacher: openai("gpt-5"),
  instructions: (sample) =>
    `Write a ${sample.metadata.type} for ${sample.metadata.audience}.`,
});

With your own prompts, the teacher is only called for samples that are missing a label in infer; pass infer: [] to make no teacher calls at all.

Metadata inference

Retrieval and evals work better when every sample says what it is. In the same call as the brief, the teacher fills in whichever of infer (by default type, audience, intent and tone; topic is also available) the sample doesn't already have. It's asked for type as a short kebab-case label, such as homepage-hero or changelog, and for the others in a few words.

Inferred values are guesses, not ground truth, so they're kept apart: in prepared.samples[i].inferred, and in each example's metadata.inferred. A sample's own metadata is never overwritten, and retrieval uses a sample's own label first, falling back to the inferred one.

Inferred type labels are free text, so a teacher may call similar samples blog-post, essay and article. Since retrieval matches labels by their words, a corpus with its own consistent type on every sample retrieves more predictably than one that relies on inference. Label type yourself when you can, and check what the teacher inferred when you can't.

Failures and cost

Know this before you run it: preparation makes one teacher call per kept sample, four at a time by default, so a corpus of 130 samples is 130 calls to a capable model. If the teacher fails on any sample, prepareVoice() rejects with a PrepareVoiceError whose failures list each sample id and its error, in corpus order, so you can fix or drop those samples and run again:

import { openai } from "june/sdk/openai";
import { copyCorpus, prepareVoice, PrepareVoiceError } from "june/copy";

try {
  await prepareVoice(copyCorpus("./fonzie.jsonl"), {
    teacher: openai("gpt-5"),
  });
} catch (error) {
  if (error instanceof PrepareVoiceError) {
    for (const { sample, error: cause } of error.failures) {
      console.error(sample, cause.message);
    }
  }
  throw error;
}

It's all or nothing: there's no partial result and no progress callback, so with a large corpus, check the teacher model and key on a few samples first. The teacher isn't deterministic, so preparing twice gives the same samples, removals and splits but not the same prompts and inferred labels.

The result

import { openai } from "june/sdk/openai";
import { copyCorpus, prepareVoice } from "june/copy";
import { fineTune } from "june/train";
import { trl } from "june/trainers";

const prepared = await prepareVoice(copyCorpus("./fonzie.jsonl"), {
  teacher: openai("gpt-5"),
  split: { train: 0.8, validation: 0.1, test: 0.1, seed: 42 },
});

await fineTune({
  model: "Qwen/Qwen3-8B",
  trainer: trl(),
  dataset: prepared.splits,
  method: "lora",
});

prepared.dataset holds every kept sample as a completion example, in corpus order: the prompt, the original text as the completion, and as metadata the sample's own metadata plus sample (its id) and inferred. So any example traces back to the writing it came from. prepared.corpus records the raw corpus's source, size (before anything was removed) and fingerprint, so a voice built from it can be saved and checked.

Reviewing a preparation

Preparation's prompts and labels are a model's guesses, and everything downstream (retrieval, evaluation, fine-tuning) trusts them. Read what it made before you write with it:

• What was removed, and why. removed should hold no surprises.
• The labels. Count kept samples by type (their own, else inferred). These are the labels requests will have to match, and a type with very few samples gives a request of that type little to go on.
• A sample of prompts. A reconstructed brief should describe the text's purpose, audience, form and length without quoting it.
• The inferred labels on a few samples. A type that disagrees with what the text plainly is will send requests to the wrong exemplars.

Deterministic splits

Splits are deterministic: 80/10/10 with seed 0 unless you pass split. They go by sample id, not by example content, because the teacher's prompts can differ between runs. Re-running preparation never moves a test sample into training, and the same corpus and seed always give the same splits on any machine.

Keep test for evaluation. Fine-tuning uses train and validation; retrieval uses every kept sample, so evaluate on held-out samples with a voice that can't retrieve them.

Keeping a preparation

The teacher work is the expensive part, so keep what it made rather than preparing again casually. A PreparedVoice holds datasets, so it isn't plain JSON. Its samples, removed and corpus are, so write those out for review and the record. June can't load a saved preparation back yet: to write in another process with the inferred labels, either prepare again (which calls the teacher) or rebuild the PreparedVoice in your app from the samples you kept, as the Copy Agent does. A voice loaded from the raw corpus works too, but retrieves with the corpus's own labels only.