The class opened with a broad question about accountability when technology causes harm. Several examples came up: algorithmic bias in recruitment tools, social media addiction patterns, AI misinformation, data harvesting in free apps. The through line is that harm is rarely the intent but it's often the outcome, and the design decisions that lead there are usually made well before anyone is thinking about consequences.
Marshall McLuhan's line — "we shape our tools and thereafter our tools shape us" — came up as a frame for this. It's a good one because it distributes responsibility. The tool doesn't act on its own but it does create conditions and those conditions shape behaviour at scale.
A few specific areas were discussed. Romance fraud and the way platforms are designed to lower emotional defenses. Gambling mechanics and prediction markets — the line between investing and gambling is thinner than most people want to admit, which is something worth sitting with given the direction of this project. Children and addiction, which is probably the most urgent version of the same underlying problem.
The China social credit system came up as an extreme endpoint of what hyper-personalised, data-driven systems can become when accountability is removed. Flock Safety and Palantir were referenced as closer-to-home examples of the same logic operating within democratic systems.
Shaping the Future of Safety, Together.
The class looked at emerging technology directions and their UX implications. Conversational interfaces and the Her comparison — the risk that an interface good enough at simulating presence starts to substitute for actual human connection. Hyper-personalisation through predictive analytics, which can feel like good service or surveillance depending on how it's implemented. AR and VR integrated into everyday life. Haptic and multisensory feedback. Biometrics in healthcare.
For the Future Living brief almost all of these are live considerations. A smart home OS that knows your patterns, anticipates your needs, and responds to your presence is operating in exactly this space. The design question isn't whether these capabilities are possible — they largely are — it's whether they're being deployed in ways that serve the person living in the space or the systems observing them. That tension is probably the most interesting thing to design around for the Pane OS concept.
<aside> 💡
This class sat with me more than most. The accountability question around technology harm doesn't have a clean answer and I appreciate that it wasn't presented as though it does. The gambling and prediction markets thread is something I'll be thinking about in relation to this project — the line between an investment platform that motivates engagement and one that exploits it is genuinely fine, and the design decisions around that line matter. The futures section at the end brought things back to possibility rather than just risk, which felt like the right note to end on. For Pane OS specifically, the hyper-personalisation and ambient computing threads connect directly to what I'm trying to build — and the ethical questions raised here are ones I want to at least acknowledge in the design rationale, even if I can't fully resolve them
</aside>