
Testing personalised print works best when you ask one clear question and give the answer a fair chance to emerge. Change the headline, imagery, offer and audience at once, and you might get a better result. But you will struggle to know why.
For businesses planning their 2027 marketing, a controlled print comparison offers a more useful starting point than a grand campaign overhaul. It helps you decide whether a particular change deserves more budget, more refinement or a polite farewell.
AI can help suggest campaign alternatives. Variable data printing can put different messages into the same run. Neither tells you which idea earns its keep. That needs a sensible test.
Start with a question you can actually answer
“Does personalised print work?” is too broad for a single campaign. Personalisation can mean a name, a relevant product, a different message or an individual destination. Each changes the experience differently.
Instead, ask something specific: does a message tailored to a stated customer interest generate more fulfilled orders than a general message to the same kind of customer?
Write the question before creating the alternatives. Then choose a primary outcome that reflects the business objective. Qualified enquiries might suit a professional service. Completed bookings might suit a venue. Fulfilled orders might suit a retailer.
The aim is to learn something usable, not simply to crown the version with the prettiest dashboard.
What a controlled comparison means
A controlled comparison usually has two versions. Version A is your baseline, often the current approach. Version B changes the element you want to investigate.
Keep other influential factors as consistent as practical: audience eligibility, offer, format, material, mailing window and response journey. If you test wording, avoid also giving one version a more generous discount.
“Change one thing” means one deliberately defined treatment, not necessarily one word. A message treatment might include a headline and supporting sentence that work together. What matters is being clear about what changed and avoiding unrelated differences.
For the same reason, keep the creative standard comparable. A professionally designed baseline versus a rushed alternative tests execution as well as your idea. Give your designer a clear test brief so both versions remain recognisably part of the same campaign.
A hypothetical personalised direct-mail test
The following is a hypothetical test design, not a Print Lord customer project or a prediction of results.
Imagine a Sussex retailer with 600 eligible customer records and reliable, appropriately usable information about stated product interests. It wants to learn whether interest-specific wording encourages more orders than a general introduction to its range.
Version A uses a general message. Version B introduces the relevant product category using one of a small set of approved messages. Both use the same overall design, product imagery, offer conditions, card specification and ordering route. If both include a name, they use it in the same way.
The retailer defines a 30-day response window, beginning from the planned arrival period. It records fulfilled orders as its primary outcome and enquiries as a secondary measure. This is an illustrative window, not a universal recommendation: a longer buying cycle may need longer observation.
The test investigates the effect of interest-specific wording among this audience. It does not establish that every kind of personalisation works for every business. Six hundred records make the example easy to follow; they do not automatically make the sample large enough for a reliable conclusion.
Allocate audiences fairly, not conveniently
Sending A to your best customers and B to everyone else would produce a comparison, but not a fair one. Existing differences could explain the outcome.
Where practical, randomly allocate eligible recipients between the versions. If important audience groups differ, randomise within those groups. In the hypothetical retailer example, each product-interest group would be divided between A and B, rather than placing all customers interested in one category into a single version.
Use roughly equal group sizes for a straightforward first comparison. Record the allocation before production, and avoid moving recipients afterwards because one group looks less promising.
Think about the unit being allocated, too. If several contacts at one business will discuss the mailer together, assigning that business to one version may make more sense than splitting its contacts. The same concern can apply to households.
These decisions affect what the test can tell you. Document them rather than quietly tidying the spreadsheet until the groups look convenient.
Make responses measurable across both versions
Give each version a distinguishable response mechanism without creating a different customer experience. That could be a campaign code or appropriately tagged destination that leads to an otherwise equivalent page.
A QR scan is an engagement signal, not a fulfilled order. Some people will type the website address, telephone or visit in person instead. Brief staff on recording campaign responses, and use the same process for both groups.
Decide how you will handle cancellations, returns, repeat orders and customers who respond more than once. Count consistently. Report recipient-level response separately from total order volume if both are relevant.
Tracking is rarely complete. Codes can be shared, and customers can forget where they saw a message. Record known gaps rather than presenting attribution as perfect.
Use only information necessary for the campaign and measurement. This test does not require uploading identifiable customer records into an AI assistant. Creative exploration and audience allocation can remain separate tasks.
Compare rates, costs and commercial value
If the groups differ in size, raw totals can mislead. Calculate the relevant response rate using a clearly stated denominator, such as fulfilled-order customers divided by recipients mailed. Record returned or undeliverable mail as well, and report it consistently.
Then consider the economics. A version that generates more orders may also cost more to prepare or attract lower-margin purchases. Compare contribution after relevant variable costs, not revenue alone, and account for additional creative, data and production work.
When reporting differences, distinguish percentage points from relative percentages. Moving from one response rate to another is an absolute change in percentage points; expressing that change relative to the baseline is a separate calculation. Both need the underlying counts for context.
No uplift is assumed here. The personalised version might perform better, worse or much the same. Any of those outcomes can improve your next decision.
Small samples need modest conclusions
Hypothetical result: A produces seven fulfilled-order customers and B produces nine from equal-sized groups. Those figures are invented solely to illustrate interpretation, not claimed campaign performance.
B leads numerically. That does not establish a dependable advantage. A difference of two customers could reflect random variation, particularly when responses are uncommon.
The sample size needed depends on the baseline response rate, the smallest improvement worth detecting and the level of uncertainty you can accept. There is no honest universal minimum for every small-business print test. For a consequential investment decision, ask an analyst to help design the comparison before mailing.
A statistically inconclusive result does not prove that the versions are equivalent. Equally, an apparent lead is not a licence to forecast the same improvement across a much larger audience.
Keep the planned response window intact. Repeatedly checking results and stopping when your preferred version gets ahead can distort the conclusion. If operational reasons force changes, record them and qualify the findings.
Use AI to suggest alternatives, not declare a winner
AI-assisted writing can help explore different ways to express an approved proposition. Keep factual claims, prices and offer conditions grounded in verified business information, and ask your creative partner to develop a coherent pair of alternatives.
A model’s confident explanation of why a headline should convert is a hypothesis. It is not customer evidence. Nor should an automated summary turn a handful of responses into a triumphant forecast.
Use the actual counts, agreed measures and documented limitations when reviewing the test. More sophisticated wording does not make weak evidence stronger.
Decide what happens next before the results arrive
Agree the likely next steps in advance. A commercially promising result might justify a larger follow-up comparison. An inconclusive result might lead you to repeat the test or simplify the campaign. A costly treatment with no convincing benefit might be dropped.
Keep a short record of the question, audience, versions, allocation, costs, outcomes and limitations. That becomes useful planning evidence for 2027, rather than another forgotten spreadsheet.
Print Lord brings over 30 years of print expertise, practical specification advice and artwork support to helping customers turn ideas into finished print. Bring the campaign objective, intended audience and proposed comparison, and discuss the production requirements before committing to a run. Statistical analysis and response systems should be scoped separately where needed.
You do not need to change everything to learn something worthwhile. Start with a clear question, make a fair comparison and let the evidence earn the next investment.
Print Lord. At your service. On brand. On time.
printlord.co.uk