The Consumer Mindset

After I graduated from university, I did the Aussie-backpacker-in-the-UK thing. My first stop was London where I arrived with what, in hindsight, was far too little money. I didn’t have any contacts there and London was much more expensive than I imagined. Had things not gone well, I may have been flying home with my tail between my legs in short order. Fortunately, I managed to pick up a job almost immediately working as an administrative assistant in a small law firm. The principal was an Australian expat who was also from Melbourne originally, which no doubt helped my chances in landing the job. The offices of the firm were in Gray’s Inn, which is one of the four Inns of Court in London and which is over six hundred years old, three times older than the country I had just arrived from. Because of the location of the Inn, I would often go on foot to carry out various tasks such as lodging paperwork at the Australian Embassy down on The Strand. It was almost the perfect job for a young man wanting to experience the sights and sounds of London.

The work itself was mundane but what was really interesting were the people you got to meet and the aspects of human psychology that were revealed by the various cases we dealt with. I was amazed by how much money people would waste on matters which clearly had no merit. We had people coming to us with cases they were never going to win often because they were the ones in the wrong. As a lawyer there is a code of ethics you must abide by in such matters so that you don’t take money for cases that have no basis in law. But in practice there is a huge grey area and there is almost always some glimmer of merit in a case; some thing where the other person was to blame. In fact, that’s true of almost all cases. Both parties are at fault but both parties think they are wholly in the right.

I was reminded of my time at Gray’s Inn recently when an acquaintance spent tens of thousands of dollars on legal fees. It was obvious from her story that she was just as much at fault as the other person. But she thought she had been wronged and it was that grievance which led her to take legal action thinking that justice would be done. I did my best to talk her out of it explaining that the only people who win in such cases were the lawyers and that she would be far better off negotiating an end to the matter with the other person directly. But she had to learn the hard way. When it was all over, she complained of all the money she had spent even though she didn’t get “justice”. Actually, from an objective third party point of view, she did get justice as she was also to blame for what amounted to nothing more than a communication problem. As Robert Plant once sang: “Communication breakdown/It’s always the same/Having a nervous breakdown/Drive me insane.” Lawyers earn an awful lot of money because of such communication breakdowns.

One of Australia’s most famous lawyers, Geoffrey Roberston, once noted that the justice system does not guarantee justice, it only provides the possibility of justice. He needed to point that out because the average person seems to think the system does guarantee justice where “justice” means prove they are right and the other party wrong. That’s rarely possible, however, for the simple reason that there are always at least two versions of justice: yours and the other person’s. But the main reason the justice system doesn’t guarantee justice is because it would be enormously expensive to do so. In the real world, systems are set up according to cost-benefit considerations. We don’t optimise, we satisfice. This follows from the 80/20 rule which states that eighty percent of the value comes from twenty percent of the cost. Every extra percent of value after that becomes more and more expensive so that the last one percent costs more than the other ninety nine and the last 0.1% more than the other 99.9% and so on. That’s why murder cases get more resources than fraud and fraud gets more resources than traffic infringements. There are no doubt all kinds of crimes that occur every day that never get addressed because the system doesn’t have the resources to attend to them. Ideally the major crimes do get dealt with but even then there is still only the chance of justice not a guarantee.

It’s a strange fact of our culture that so few people understand this. People seem to think systems are these flawless machines that deliver a fixed result every time where the result just happens to be what they want. They think that if somebody does you wrong, the justice system will make it right. They think that if you get sick, the medical system will bring you back to perfect health. Actually, the justice system and the medical system are there as a safety net when things go wrong. The best thing you can do is avoid them. If you never have to see a lawyer or a doctor in your life you can consider yourself very fortunate. And you should try and make it so you do avoid lawyers and doctors. You can avoid the justice system, especially in business dealings, by making all expectations clear upfront and signing agreements and contracts that stipulate clearly what people are agreeing to. It’s far cheaper to get the lawyers involved at the start than at the end. Same with the medical system. Keep your health in order, eat well, exercise, practice basic hygiene and you will avoid the medical system as much as possible. That’s the best strategy. But many people seem to think that they must go to the doctor in order to be healthy even for things which are obvious lifestyle problems like high blood pressure.

No doubt there are many factors that have led us to this strange position but one that I think is a big part of the issue is that people apply the consumer mindset to such systems. The consumer economy works by providing an item that does a fixed thing for a fixed price. You buy a toaster for $30 and it cooks your toast. You buy a microwave for $150 dollars and it warms your food. Simple, linear and reliable. Of course, the consumer economy itself relies on an enormously complex system of mining, manufacturing, transport and electricity generation but all that is hidden from the consumer. With the rise of consumer society, people have learned to think in a linear, simplistic fashion. They then apply that model to domains where it doesn’t belong. They think that they can just pay a lawyer to get “justice” or a doctor to get “health”. But the legal system and the medical system are not the consumer economy. They are irreducibly systems and in systems there are no guarantees, only probabilities. They should be used as a last resort but that’s not the way that people think about them these days. Thus, the medical system and to a lesser extent the legal system have come to be seen through the consumer mindset.

When the system doesn’t deliver the desired outcome, some people blame the practitioner. Lawyers already have a low reputation for this reason but it wasn’t long ago that doctors did too. We used to call them “quacks”. My grandmother always used to say “we better get you to the quack”. Doctors and lawyers were seen as necessary evils. They didn’t guarantee you an outcome but they did guarantee that you had to pay them. Fancy offices at Gray’s Inn don’t pay for themselves after all. On current trajectory, I wouldn’t be surprised if we again start referring to doctors as quacks in the near future. That won’t be a bad thing. It will be a recognition that systems don’t guarantee outcomes, that self-responsibility is the best bet and, to use another favourite phrase of my grandmother, “life was never meant to be fair” (where “fair” means almost exactly what my acquaintance meant by “justice”).

Science vs Science Fiction

In Plato’s Republic, the philosopher takes a very hard line against the arts in general and the poets in particular, even going so far as stating that Homer should be banned by the philosopher kings ruling over the ideal state. Plato’s main objection to poets was that they are just imitators and imitation is devoid of knowledge. More specifically, the arts engage the emotions and not the higher rational faculties. They throw the Platonic psyche out of balance at the individual and the societal level. Were he to be transported to 2021, Plato would be horrified by the sheer volume of “poetry” we consume via television and the internet in modern society. Even if all our storytellers were as good as Homer and accurately imitated life in their art, we would be out of balance in Plato’s eyes by constantly stimulating our emotional and imaginative faculties without subsequent stimulation of the reasoning faculties. As it turns out, our society provides quite a lot of evidence to suggest that Plato was right. Our public discourse runs very much on emotions and very little on reason these days and it’s entirely possible that this tendency is in direct proportion to our proclivity to watch movies and tv shows rather than engage the rational faculties. It doesn’t help that the propaganda machine formerly known as the news media also indulges in the blatant fabrication of reality. If imitation was bad enough in Plato’s eyes, what would he make of the fabrication and distortion that is now business as usual?

As somebody who enjoys both writing and reading stories, I would disagree with Plato’s objections to the artform primarily on grounds that stories and art in general are a bridge to the unconscious mind and the unconscious mind can reveal truth. Of course, the unconscious mind does not exist in Platonic psychology. His psyche has reason, spirit and emotion and in The Republic he extrapolates this structure at the individual level to society at large. Thus, the philosopher kings represent reason and should rule. The armed forces represent spirit and should be subordinate to reason. The rest of society represents emotion and this should be subordinate to both spirit and reason. If we were to introduce the unconscious, and in particular the collective unconscious, into the psychic equation and give it a prominence of equal weight to reason, then the poets, storytellers, priests and others who were concerned with it would have a great responsibility to ensure that the symbolic representations of their craft were faithful to whatever truths were to be had through the unconscious. As a storyteller, I believe that to be true. It’s the duty of storytellers to make sure a story is accurate and this goes for the plot, the psychological and biographical accuracy of the characters and even the symbolic meanings of the story. When all these are taken care of, the story resonates at multiple levels at once in much the same way that harmony functions in music.

It’s for this same reason that I tend to be very critical of stories and movies where the author or screenwriter gets it wrong. Let me give one example that’s always annoyed me from the movie Gladiator. Those who have seen it will remember the scene where Maximus is arrested and taken by a troop of praetorian guards to be killed in the forest. He manages to break free and kill the first few guards. There are a couple of others who are on watch at a distance and don’t know what has happened. He kills the first by throwing a sword from behind, a very low risk technique. Then he kills another. There’s one guard left; one man standing between Maximus and freedom. This guard has managed to remain blissfully unaware of everything that has happened. His attention is off in the distance. Maximus is standing behind him with sword in hand. We’ve already seen Maximus kill one guard by throwing a sword from behind. He could easily do the same with this one. Alternatively, he could get the horse of one of the other guards and ride away without even bothering to kill the man. Both are zero risk options which get him what he wants. Instead, he challenges the guard to a duel where he is at a significant disadvantage by not being on a horse. He kills the guard buts gets injured in the process and the rest of the movie unfolds from there. This scene makes sense as a plot device. It gets the story where it needs to go. But it doesn’t make sense in terms of characterisation. Are we really to believe that Maximus, Rome’s greatest general, who has just shown great discipline and fortitude leading his troops into battle, is going to take a completely unnecessary risk that leaves him at a significant disadvantage in a fight? I don’t think so. He would have thrown the sword, killed the last praetorian guard and ridden away on his horse. Nevertheless, the error is minor and I’m sure most people watching the film didn’t even pick up on it. Gladiator is an action movie, after all. People are not watching it for an in-depth psychological analysis. Sometimes, however, errors like this are revealing about the culture. I find science fiction to be a rich source of such errors which are interesting to the extent that they reveal something about our culture’s understanding of science.

Robert Heinlein defined science fiction as “realistic speculation about possible future events, based solidly on adequate knowledge of the real world, past and present, and on a thorough understanding of the nature and significance of the scientific method.” In other words, science fiction should be accurate in its depiction of science in the same way that a story’s plot should be accurate in relation to its characters. In Plato’s language, it should imitate reality. We’ve all had the impression while watching a movie or reading a book that the character “would never do that”. In that case, the storyteller has failed to marry the plot and the characterisation. In the same way, we might have the impression while watching a science fiction move that “science doesn’t work like that” or “that could never happen [because it breaks the laws of physics]”. An example of this is The Matrix. In the movie, we are told that humans are being farmed for energy because the sky was blacked out and the sun blocked. This makes no sense from a thermodynamics point of view. Even assuming you could keep humans alive in such a world, how are you going to feed them? What sort of plants are growing when there is no sunlight to photosynthesise? And how much energy does The Matrix itself use up just keeping the humans distracted? If you were a smart AI, you’d be better off capturing whatever energy is still coming from the sun directly rather than running it through human beings. That would be more energy efficient and, let’s face it, humans are a pain. They have a nasty habit of not doing what they’re told, even when they’re stuck in little pods in the sky. Better off to get rid of them and do whatever it is that AIs like to do with their time. So, this plot device doesn’t work within scientific theory. Another common problem in science fiction is the portrayal of scientists in movies.

Let’s take just one example that bugged me so much I stopped watching the movie: the film Sunshine released in 2007. The story is set in the year 2057. The sun is dying and the earth is getting too cold to live on. Humans come up with a plan to nuclear bomb the sun back to life. They have already sent one spaceship to do the job but communication with it was lost. They send a second ship and that is where the movie begins. While en route to the sun, the second ship establishes communication with the first. They have a choice to carry out the mission as planned or deviate and unite with the first ship. They decide to try the latter. The ship has a supercomputer on board which handles the calculations but, for reasons not explained in the story, they have the ship’s mathematician override it and do the calculations himself. The story makes a big fuss about how difficult the calculations are and how much pressure the mathematician is under to get the done before it’s too late. He makes a mistake and the story goes from there. What is the error here? The error is that you would never let a human calculate by hand when you have a computer there to do the job instead. One things computers undoubtedly do better than humans is calculation especially when there is a time constraint and high pressure situation. Within the plot of Sunshine, it is no surprise that the mathematician made the error. The problem is that nobody on the ship should have allowed it to happen. This is supposed to be a team of scientists and smart people. They should have known better.

So, this is an error just like the one above in Gladiator. Something happens in the story that would not happen in real life. But I think this error reveals something about our cultural understanding of science. We have the stereotype of the genius scientist or mathematician and we think the genius lies in calculation. This ties in with the whole issue of IQ testing where it is assumed that ability to manipulate symbols quickly and accurately is the sine qua non of intelligence. Ask the average person why Einstein or Newton were so smart and chances are they will say they were better at maths than others where “better at maths” means able to calculate things that other people were not. That’s kind of true except the real difference lies not in the calculation ability but the ability to re-define a problem so that it can be calculated or invent new techniques that enable calculation. It does not lie in the ability to do the calculations but that’s what the mathematician in Sunshine was doing; pretending to be a computer. That’s the first problem.

The second problem with the stereotype in Sunshine is the idea of the solitary genius. The ship has just one mathematician aboard and he has to work alone to solve the calculations. In reality, the whole point of science is that others are there to help check your work. You have to explain your methodology and your results and let others reproduce them. The Apollo space program had an estimated four hundred thousand engineers and technicians working on it and a huge part of that effort was in checking and re-checking each other’s work to find mistakes. Even Newton said he was standing on the shoulders of giants. But in our culture, we have the idea of the solitary scientific hero. The solitary hero was already common in western culture and specifically US culture prior to science fiction. This is the lone rider motif. What we have done with much of science fiction is map that motif onto science where it does not belong.

Most scientific breakthroughs are based not on calculations by super high IQ individuals but by one of two primary methods: 1) an imaginative/intuitive re-definition of a problem or theoretical framework; 2) a stochastic process (read: blind luck) that leads to a re-definition of a problem or theoretical framework. These are not mutually exclusive and in fact one almost certainly leads to the other by opening up new areas of exploration which then force a re-defining of theoretical frameworks. In real life stories of science, we find both of these elements. Let’s take a couple of examples that are very topical right now as they relate to vaccines.

Louis Pasteur is credited with the invention of the attenuated vaccine. At that time, trying to prevent mass death by viral disease among livestock herds was the main driver for vaccine research. Pasteur had been working away for years on the problem and making no headway. On the last day before the traditional August summer holiday in France, one of Pasteur’s lab assistants was supposed to do the processing on the latest batch of trial vaccines but forgot. When he returned from holiday he realised his omission but, rather than own up to it, simply injected the chickens with the batch he had left untreated expecting them to die like all the other lab animals previously. But they didn’t die. They got better. For the first time, the test seemed to work. The lab assistant told Pasteur what had happened, they investigated and did more tests and eventually came to the attenuated vaccine. That kind of luck is common throughout the history of real science but is absent in our science fiction. Note that Pasteur is credited with the invention of the vaccine when, in reality, you could argue that it was his lab assistant who was at least partly responsible.

Teamwork is also not emphasised in our cultural depictions of science though it is crucial to real life science. Take the story of the invention of the mRNA vaccine as told by Robert Malone. It features a little bit of a luck and also teamwork as it was one of the scientists on the team who insisted on doing a negative control test when everybody else was thinking of other things that helped the evolution of the process along. Malone is very happy to point those facts out as any real scientist would. That is how science works in real life. As Woody Allen said, showing up is 80% of success. Show up each day and do the work and “luck” falls into your lap. That’s how vaccines came into being: luck and teamwork. Of course, that doesn’t make for a dramatic movie or a good story. Neither does the other process by which scientific breakthroughs are made. These are the Eureka moments where a scientist has a sudden intuition that reveals the solution to a problem. One minute you’re taking a bath and the next moment you have the answer. That doesn’t make for good film either. Almost by definition, storytellers must fabricate the truth in relation to science in order to make it fit into the structure of good fiction.

I could go on with my list of gripes about science fiction. One day I might do a whole post about the movie Interstellar which actually made me angry to watch. It takes some of the tropes I have mentioned here and added others which reveal something about modern culture that is directly relevant to current events. In any case, the problem is that such art is not even a good imitation of reality. To return to Plato, our poets do not even imitate. They fabricate and distort. In doing so, they are creating and re-creating the underlying mythology of the culture but what we now know is that the mythology, through the collective unconscious, has a real effect in the world especially in a society which consumes such a huge amount of myth and fiction relative to reality. It’s tempting to agree with Plato that banning it all would be the best idea. Imagine the current world if somebody flicked a switch and there was no more television, cinema or Netflix. It’s hard to see that as being anything other than a godsend right now. How many of the current social neuroses are fed through that apparatus and would promptly disappear if the apparatus went away? Maybe Plato was onto something.

Is agile software development a living design process?

I recently wrote a post outlining some principles for doing Living Design Process, a concept developed by permaculturist, Dan Palmer. These were inspired by my experience with a house renovation and permaculture-inspired garden project.

As I was reading back over the post, I realised that these principles were familiar to me from my work in agile software development.

In theory, agile software development should be an example of a living design process. In practice, it’s not. I’ll use the 7 principles from my blog post to explain why.

Principle 1: Embed yourself in the context

We do this pretty well in agile software development. We have co-located, cross-functional teams that share knowledge. Teams usually struggle on two points.

Firstly, we often don’t have the high level business context i.e. why is our company spending money in this area. This is usually because the company itself doesn’t have a coherent strategy. It’s exacerbated by the fact that (upper) management is not part of the cross functional team and therefore we don’t have direct contact with them.

Secondly, we often don’t have the customer or user context. Sometimes this is because the customer doesn’t exist yet.  Sometimes it’s because we simply don’t have access to them. The most fulfilling projects I have worked on were when we had direct access to the customer.

Principle 2: Have high level goals, not specific ones

A high level goal might be something like “provide customers with an easy way to sign up for electricity connection”. Building software to achieve that goal is already a solution. A very expensive solution.

I have been on a couple of projects where a non-IT solution would have been preferable but we’d already assembled a software development team and couldn’t change tack.

Some companies are spending more time on product validation before the IT build begins. But most companies just start writing software. They move straight into specific goals and lose sight of the bigger picture.

Principle 3: Start small and iterate

We are getting better at this. There’s movement away from monolithic systems and product development is experimenting with ideas around MVP and build, measure, learn.

However, iterating implies starting with a functional first attempt and then re-visiting it to tweak, strengthen and harden it based on feedback. This re-visiting happens very rarely. It is perceived as “re-work” and seen as waste. As a result, there is a stigma attached to it.

Most agile projects have way too many features in the pipeline and not enough time to deliver them. We are too busy churning out new features to properly iterate on the solution.

Principle 4: Go Slow

This whole concept is anathema to most software delivery teams. We have “delivery managers”. We track velocity and cycle time. We use kanban boards to make sure that we are maximising flow. Everybody wants to know how we can go faster, not slower.

This is built into the DNA of corporations. Budgets are drawn up and expenditure is fixed in advance. Each project has a fixed amount of time and money before it even begins. You simply don’t have the luxury of releasing something, taking feedback and then reacting.

Principle 5: Maximise Optionality

Despite some good steps towards reducing complexity such as having “2 pizza teams”, the average software development project is still too complex. As a result, we are constantly bombarded by events that threaten delivery of the project. These include technical considerations but also internal organisational issues.

This results in a defensive mindset. There’s no room for keeping an eye out for opportunities. There’s no time to learn.

Principle 6: Embrace Randomness

Organisations have their own political structures. People have to lobby and compete for resources to get a project underway. In order to do that they have to tell a compelling story.

Randomness changes that story. This causes a political headache for whoever is advocating the story. Good middle managers quickly learn how to spin these changes and adjust the narrative. But this takes time and energy and you only get a couple of chances to change the story. After that, you appear as wishy-washy or incompetent.

Randomness also becomes harder to incorporate the more code you have written. If you change your mind and force people to throw away their work and start again, you burn some goodwill.

As a result of these dynamics, teams are averse to randomness.

Principle 7: Find the right people

In most organisations, teams are put together on an ad hoc basis.  (Note: there are a few organisations that have experimented with allowing employees to self select onto a team.)

As team size is still too big and turnover is quite high due to holidays and people leaving the company, the odds of putting together a genuinely high performing team are very low.

The reason our teams are bigger than they need to be is to hedge against the possibility that two or three people quit the company at the same time and thus threaten the delivery of the project. Companies prioritise stability and predictability over high performance.

In summary, agile software development falls short of being a living design process on the following grounds:-

  • We don’t have a holistic approach. Even when teams are told about the business goals, they are not actively involved in shaping those goals. You are there to deliver software, not to develop a business. The business still thinks of IT as a delivery function. In Jeff Patton’s words, this is the vendor-client anti-pattern.
  • We don’t have the high level goal in mind. By the time a team is formed, somebody else has determined both the business focus and the high level technical implementation. Individual team members are not encouraged to think about these things.
  • We don’t iterate properly. The focus is on getting as many features in before you run out of time and money. This is called a Feature Factory.
  • We often have fixed dates which prevent an open-ended exploration of the problem space.
  • Teams are still too big and are randomly put together.

Many of these issues reinforce each other. For example, because teams are too big there’s more complexity and more expense. Because it’s complex, you are always on the back foot and in a defensive mindset.  Because it’s expensive, you run out of time and money quicker. Because you’re running out of time and money, you can’t take the time to explore the problem space. Therefore, you can’t capitalise on opportunities. Therefore, you can’t pivot properly in the face of new information etc.

Some of these problems are hardwired into the organisation. The corporate governance structure imposes specific limitations on how funding is granted, who is to be held responsible for decisions etc. The most interesting companies in this space such as Gore and Semco have addressed this issue by changing their governance structures.

At one of my earlier jobs, the CIO got up in front of the IT department and told us that “the business would never become agile”. Many years later it’s increasingly obvious to me that this is where the problem lies. We have pushed as much as possible from the bottom up. The final steps towards a “true agile” are now at the organisational level.