Book I · The Next Evolution · How To · No. 4
Build Trust Into Unseen Infrastructure
For when the system that decided about you won’t say why, and neither will anyone you can actually reach.
Making the invisible visible — not just technically, but ethically — so people can question, understand, and challenge the systems that shape them.
The letter says the mortgage application doesn't meet current lending criteria. Nothing else. The broker doesn't know why. The call centre reads from the same three lines everyone else gets. Somewhere behind the decision sits a scoring model that nobody in the building can actually explain, trained on data nobody in the building chose.
That isn't a broken process. It's infrastructure working exactly as built — just built without anyone answerable for what it decides. The card's own argument starts there: the most powerful systems are usually the invisible ones, shaping outcomes nobody gets to question because nobody can see them operating in the first place.
The card holds five practice areas across two groups: what you design and verify into a system before it ever makes a decision — transparency, user control, outside audit — and what you build around it afterwards, so accountability doesn't depend on goodwill. Written for whoever builds, deploys or governs systems like this, though anyone who has waited for an explanation and received a form letter instead will recognise exactly what it's asking for.
It comes from The Next Evolution, where the fuller argument lives. The card stands on its own — you don't need the book to use it.
Print it, or send it to whoever is about to ship the next system that will decide something about someone who'll never see how.
And here's the question I'd leave you with: next time a decision arrives with no explanation attached, do you actually know who — or what — made it, and whether you have any real way to ask?
Build Trust Into Unseen Infrastructure
Design & Verify
Design for Proactive Transparency
- Communicate clearly how systems work. Explain what data they collect, how it's used, and who benefits — without jargon or legalese.
- Prioritise explainable AI over black boxes. Choose systems that can explain their decisions and outputs in terms a person can actually follow.
- Adopt open standards and protocols. Open-source software, open APIs and standardised formats allow real scrutiny and interoperability across systems.
- Log every critical operation and data flow. Immutable audit trails let the record be checked and verified after the fact, not just trusted in the moment.
Foster User Trust and Control
- Collect only the data a system actually needs. Delete it once it's no longer necessary for the system's stated purpose.
- Give users granular control over their own data. Let people choose, specifically and knowingly, what's shared and with whom.
- Design for understanding, not just use. Interfaces should let people challenge and manage their interactions with a system, not just operate inside it.
- Build privacy into the architecture from day one. It has to be structural, not a policy bolted on after the system already exists.
Promote External Validation and Auditing
- Bring in independent third-party audits. For critical systems, make external review of algorithms, data governance and security mandatory, not optional.
- Pay people to find what you missed. Bug bounties and responsible disclosure programmes turn outside scrutiny into an asset, not a threat.
- Measure fairness against a shared benchmark. Industry-wide metrics let a system's ethical performance be compared, not just asserted.
Govern & Sustain
Establish Robust Accountability Frameworks
- Name who owns each system, end to end. Design, deployment and ongoing operation each need a clear owner — a blame vacuum helps no one.
- Assess the impact before and during deployment. Regular algorithmic impact assessments should weigh the societal, ethical and human rights consequences, not just the technical ones.
- Back independent bodies with real power to intervene. Regulators, ombudsmen and oversight committees only help if they can act on systemic failure, not just observe it.
- Give people a real route to redress. Anyone affected by an automated decision needs a clear, accessible way to complain, appeal and be heard.
Cultivate an Ethical Innovation Ecosystem
- Teach ethics alongside engineering, not after it. Training should weigh societal impact as seriously as technical capability, in design and management too.
- Stand up an internal ethics review board. A red team that looks for harm before launch is worth more than one that explains it afterwards.
- Protect the people who raise concerns. Strong whistleblower protections make it safe to flag unethical or harmful practice before it ships.
- Reward trustworthy design, not just fast design. Grants, certifications and procurement preferences can make responsible technology the commercially smart choice too.
The greatest technology of all is still us.
From The Next Evolution · © Neil Catton · Filed under design and leadership
Read it alongside
Also from The Next Evolution · No. 3Assert Ownership of Your Digital SelfFor when you realise the version of you that platforms hold is more detailed, and more valuable, than the one you’d choose to show anyone.
Also from The Next Evolution · No. 5Co-create Experiences of Efficiency and EquityFor when a service gets faster for the people who already found it easy, and nobody asks who got left behind.
Also from The Next Evolution · No. 2Become a System ShaperFor when the device decides before you do, and calls it a courtesy.