I finished reading Careless People, Sarah Wynn-Williams’ account of her years at Facebook, while I was still in Sydney recently, and – despite the uplift to my spirits that always comes when I spend time in that wonderful city – I found myself quite unsettled. This is especially the case because I am currently immersed in supporting the school team here at Dalton School Hong Kong in the educational equivalent of ‘pre-season training’; it is (as I have said many times before, and will continue to say!) a real privilege to be a Board Member of this future-focused, progressive, deeply values-led school, and to play a small part in helping children and young adults shape their futures.
And so this is where my learnings from Careless People are sitting very uneasily with me, because they speak to a world in which our children will, if we and they are not careful, be manipulated by powerful forces whose interests may have very little to do with their wellbeing. The key imperative that lodged itself firmly in my mind as I turned the final page and closed the book is that we have an urgent and core responsibility to enable children to understand what is happening around them, and to learn how to protect their independence of thought and action; or – at the risk of sounding over-dramatic – some of the precious core tenets of our civilised societies are fundamentally at risk.
Allow me to explain … in Careless People, Wynn-Williams describes beginning her career at Facebook with an idealistic belief in the potential of the company to connect people across borders, widen access to information and perhaps even change the relationship between governments and those they govern; she sees something important happening and wants to move closer to it, partly to understand it and partly because she believes that she might help steer it well. What emerges, though, is the extraordinary speed with which a company created to connect university students became something approaching a global political force, its leaders travelling the world to meet presidents and prime ministers, and its decisions affecting societies about which those making them sometimes appeared to understand remarkably little.
The consequences were far from theoretical: in Myanmar, for instance, where Facebook became for many people virtually synonymous with the internet, the company has acknowledged, following an independent human rights assessment, that it did not do enough to prevent its services being used to foment division and incite offline violence, with devastating consequences for the Rohingya. In 2016, according to Wynn-Williams, Facebook employees were embedded with Donald Trump’s presidential campaign, helping it to exploit the extraordinary sophistication of the platform’s advertising tools to identify and reach particular groups of voters. These examples expose different aspects of the same unsettling reality, namely that Facebook possessed both extraordinary reach and extraordinarily detailed knowledge about its users; what was happening was that those tiny, apparently inconsequential traces that we leave as we wander around the digital world – the links we open, the things we linger over, the people we follow, the posts that anger or amuse us – were being assembled into an increasingly intricate portrait of what mattered to people, and what might move them. That knowledge was then being used to decide what they saw next … and this is where influence begins to blur, disconcertingly and almost imperceptibly, into manipulation.
Political campaigning and advertising have always sought to persuade us, of course, but historically we could at least see the billboard by the roadside, read the newspaper advertisement and know that someone was trying to influence our behaviour; micro-targeting is altogether more intimate and elusive, because I may see something designed specifically to provoke my particular fears or hopes, while you, sitting beside me, are shown an entirely different message, and neither of us knows what the other has seen. We can therefore inhabit increasingly personalised versions of reality while continuing to believe that we are simply looking at the world as it is.
And then there is power itself, and the question of what happens when those who control these extraordinary systems are challenged. Since the publication of Careless People, Meta has used an arbitration agreement to restrict Wynn-Williams from promoting the book or making statements it regards as disparaging; Meta says that the book contains false and defamatory claims, while Wynn-Williams has challenged the restrictions. At the Hay Festival in May, this led to Wynn-Williams sitting on stage for an hour during a discussion about the power of technology, but in complete silence, apparently (on legal advice) unable even to nod or shake her head. There is something almost unbearably symbolic about a woman who has written a book about power, influence and the suppression of inconvenient truths sitting physically present while others discuss these issues, yet unable to contribute a single word. (She received a standing ovation – rightly so.)
And all of this matters even more because we are now hurtling headlong into AI, where the possibilities for personalised influence are becoming considerably more sophisticated. Social media has largely determined what information is placed in front of us; AI will increasingly interpret it for us, respond to us, advise us, teach us, anticipate what we need and perhaps come to understand how best to persuade us. I use AI extensively and am enormously excited by its possibilities, particularly in education; I can see enormous potential for AI to widen access to knowledge, foster creativity, deepen learning and give children opportunities that previous generations could scarcely have imagined. But Careless People is a timely reminder of what happens when our enthusiasm for what technology can do races ahead of our consideration of what it should do, and when the power to make those decisions becomes concentrated in remarkably few hands.
This brings me back to schools, to educators and to parents, because I increasingly think that one of our most important tasks is to help young people develop a kind of intellectual alertness and self-defence. Digital literacy can no longer mean simply knowing how to use technology safely and competently; children need to understand that what appears on a screen has been selected, ranked, generated or promoted for a reason, that apparently neutral systems carry the assumptions and interests of those who design and control them, and that something which provokes a powerful emotional response deserves perhaps more scrutiny, rather than less. We need them to develop the instinct to wonder: why am I seeing this; who wants me to see it; what do they know about me; what might they want me to think or do; what evidence sits behind it; and what might I encounter if I deliberately went looking elsewhere? This is not about making children suspicious of everything, nor about frightening them away from technology which has the potential to enrich their lives enormously; rather, it is about giving them sufficient knowledge, curiosity, discernment and agency to remain authors of their own thinking.
We talk a great deal about preparing children for the future in terms of the skills they will need to flourish within it, and rightly so; but I am certain that we need to think equally carefully about what they will need in order to withstand it, question it and, where necessary, have the confidence to refuse to be shaped by it. Our children will inhabit this world long after we do, and as educators and parents, we simply cannot leave the ethical boundaries entirely to the people building the technology, however clever or well-intentioned they may be. Together, we have a genuine responsibility to equip the next generation to look closely, ask awkward questions, and hold on firmly to their own agency.
After reading Careless People, I am utterly convinced that we cannot afford to be careless about that.
