Discover more from Adaobi’s Newsletter
No one software should have all that power
A quick note.
I recently read this blog post and it completely changed how I consume and produce content. To keep it brief, it has changed my motivations for writing, and therefore what I will write about. I will now be writing for 100% myself, with the aim of bringing more clarity to my thinking (although thinking = writing). What that means for you, the reader, is that some of my posts (maybe all) will be extremely boring.
Do with this information what you will.
A friend of mine recommended I read Manna, a dystopian book about two alternative futures of humanity. In my opinion this book was very poorly written and I don’t recommend others read it. However, I will be focusing on the message it was trying to deliver (at least what I perceived the message to be).
In Dystopian future 1, Manna was a software that measured, managed and predicted an individual’s (work) performance. Because of this, it was able to hire, and fire, everyone without human input, and eventually replaced most people with robots. As a result of Manna, there was extreme wealth inequality (the large majority of people lived on welfare) and sadness. Given my own interest in measuring people, I will only critique the measurement aspect of Manna, and that, I can say, was honestly a big bag of bullocks.
Let’s assume the worst and discover the software Manna exists. Are we saying that the decisions Manna makes, humans wouldn’t also make? Isn’t the whole point of software that it is a reflection of what we already do, but at scale? Decisions about people at that scale, with that level of potency is very hard to ignore, unlike everyday human judgement. It’s this scale and potency of bias that force us to acknowledge our own and improve them.
Ironically, we could start measuring and analysing people today, but we choose not to. Our justification being that we are “protecting individuals”, which is understandable, but not entirely true. When our reality today consists of:
Individuals getting access to opportunities based on who they know
Individuals getting access to opportunities based on a limited supply of credentials (done so intentionally)
Individuals getting promoted based on likability and familiarity
Hiring decisions influenced by an individual’s mood
how can we say with confidence that we aren’t already harming individuals. We are so worried of what could be, that we forget what is.
Putting my frustration with the book’s messaging aside, as someone who tried to build a similar product, it did make me reflect more on the ethics of measuring people. To measure people is to influence social order, and that is a lot of power for anyone to yield. I eventually settled on the fact that ““No one man should have all that power”. I’m wise enough to know that my own view of the world will not be a perfect fit for everyone, and that everyone, no matter how pure, is prone to corruption.
That said, I still believe this system should be built, I just worry about who will build it (that it will be built is something I believe is inevitable). Most people tend to build in their own image (e.g Reid Hoffman is very network driven, and LinkedIn reflects that), so the uncertainty around this does make it a little scary, but again, I don’t believe a dystopian outcome is inevitable.
To help prevent this, here is what I will be doing on an individual level:
Train myself in ethics (video and book recommendations welcomed!)
Intentionally hire an extremely diverse team when I rebuild this company or build my next company
Surround myself and befriend all types of people (particularly those I agree on very little with)
As for what I think the world needs to do, I’m still not sure.
I don’t believe having “good governance” solves this problem, as the idea of “fairness/morality/ethics” being left in the hands of a few still doesn’t sit well with me, since they too are still prone to corruption - see every government that has ever existed. The majority’s (everyday people) input is needed somehow. I don’t know how but I think this is what is needed.
If you are like me you might be thinking this is a bunch of fluff, but for now it’s the best we have got - so I encourage you to come up with better solutions. Plus, it’s better to start with something and iterate into a better solution, than to wait for some grand solution that may never come (Lean Start-up fings init).
In case you were curious, Dystopian future 2 consisted of extreme abundance of wealth for everyone, extreme happiness, everybody’s jobs being automated and people living full time in virtual reality. Make of that what you will.