I ran a taste harness on a decision I had delegated

By the time I ran this site through Preference Harness, the decision was already over. I had shown three treatments to an agent, delegated the final choice to it, and let it implement the winner.

I reconstructed that sequence afterward. The tool accepted the history. It refused the inference I wanted to make from it: the dossier is valid, the preference contract is incomplete, and the verdict is NO VERDICT.

That distinction sounds fussy until an agent starts carrying its own decisions forward as if they were yours. A delegated choice becomes an accepted example. The example becomes a rule. A few rounds later the system can explain your taste back to you, complete with evidence that it produced itself.

I built Preference Harness to stop that laundering step.

The expensive version of “I do not like it”

This started with an honest and nearly useless brief:

I do not know why, but I do not quite like it.

I had put finished redesigns in front of an agent and rejected them. One rejection produced something actionable: a new page felt like another skin on the same structure. The others were real judgements with nowhere specific to land.

The useful turn came when the unit got smaller. Instead of another homepage, I saw a page of named techniques running on my content: grain, dithering, lit borders, text motion, magnetic hover, scroll-driven reveals. It was much easier to point at one.

That did not prove a theory of taste. The successful round changed the subject, the size of the unit, the number of options, and the presentation at once. It only showed me a cheaper way to ask the next question.

This is where my earlier article on harness engineering stops. A harness can check a known rule. It can preserve accepted examples, run screenshots, enforce contrast, and catch drift. It cannot decide that a preference belongs to me before I have made the decision.

Preference elicitation is the step before the invariant.

It could not be one taste skill

My first instinct was to make a good “taste” skill. That fell apart as soon as I tried to name what it would judge.

A public landing page has a reading path, a promise, and a conversion goal. A product screen has tasks, states, errors, and recovery. Technical prose has claims, pacing, and source discipline. The same density or motion can be useful in one and destructive in another.

So the repository is a suite:

• preference-calibrate narrows one vague rejection into a comparison and a scoped rule;
• preference-marketing-web applies a matching rule to public and editorial sites;
• preference-product-ui applies one to task-oriented interfaces;
• preference-verify checks an exact revision without merging preference with correctness;
• prose-verdict returns PUBLISH or HOLD for a complete piece of public writing.

The router requires an exact scenario and axis. There is no generic fallback. Native mobile, data visualisation, developer tools, and motion are unsupported until they have their own tasks and evidence.

That refusal is part of the product.

The landing comparison was a useful failure

I used the Open Source section of this landing as a live case. It had four projects in a two-column grid and each one looked like a familiar dark portfolio card. I prepared three treatments in the full page context:

• A changed the geometry into full-width rows;
• B kept the grid and changed the typographic hierarchy;
• C kept the grid but replaced the card material with flat separators and a left seam.

The comparison was useful for design review. It was not a controlled preference trial. The alternatives changed different factors, so a single choice could not support one causal rule.

Then I broke the more important condition: I handed the decision back to the agent.

The agent chose C and implemented it. It also found two objective defects shared by every option: the contact email clipped at 320 pixels, and the fixed back-to-top control covered content in the equivalent 200% reflow view. Both were repaired. The final section passed keyboard focus, reduced motion, mobile width, and production build checks.

Those are correctness and task-fit results. The preference lane stayed empty.

I ran the case through Preference Harness only afterward. The dossier classifies the three manifests as reconstructed_history, marks their claim use as diagnostic_only, and records the chosen option as model_review. It creates no preference rule.

The dossier, validation result, and public report are committed and reproducible. The result is:

1234valid: true
contract_ready: false
release_ready: false
verdict: NO VERDICT

Adding Preference Harness as a fifth project does not turn it into a held-out test. I knew the treatment before adding the item, and I made no prediction about the new artifact before seeing it. Calling that transfer would be another story written after the result.

What NO VERDICT protects

The obvious demo would end with a neat rule such as “Petru prefers flat seams to cards.” I do not have that evidence.

What I have is narrower:

• the old cards bothered me enough to ask for another pass;
• three alternatives exposed concrete differences;
• I delegated the final decision;
• the chosen treatment works by the objective gates we ran;
• no live human choice supports promoting it as my preference.

Preference Harness keeps those facts separate. Responses record whether they came from a live person, a historical note, a deterministic fixture, or model review. Rules carry scope and exclusions. A held-out prediction must exist before the response. Corrections append new records instead of rewriting the old ones.

Correctness, task fitness, and preference get separate assessments. There is no overall score that lets a green build compensate for a preference mismatch, or a human preference excuse broken keyboard access.

The public report also removes human text that was not approved for publication. Provenance fields are still declarations rather than authenticated identity, and a “controlled” trial means its declared factor manifest is controlled. The CLI does not compare DOM trees or pixels to prove that nothing else changed.

Those limits are in the README because they are where a useful tool turns into a persuasive story about itself.

The next test

The next attempt is deliberately less dramatic:

1. choose one scenario and one unresolved axis;
2. change one declared factor across neutrally labelled alternatives;
3. let the decision-maker choose, reject all, or abstain;
4. write a rule with an explicit scope and exclusions;
5. predict the result on a hidden artifact before showing it;
6. assess correctness, task fitness, and preference independently.

The current repository has 39 contract tests, six supported synthetic cases, one explicit refusal case, five portable skills, reproducible CLI bundles, and CI on the maintained Node.js 22 and 24 release lines. That proves the machinery behaves as documented. It does not prove that the suite improves human preference alignment.

You can inspect the routes before installing anything:

1bunx skills add Arakiss/preference-harness --list

I wanted a tool that could turn taste into a rule. Instead, it gave me a reason not to write one.

That is less satisfying than a successful demo. It is more useful than a false preference.