The Black Swan, Nassim Nicholas Taleb (2007)

This is a book that I almost didn’t read. Like The Long Tail or Here Comes Everybody, for instance. Both books I own but don’t feel the need to read, feeling that I've already having experienced much of what lies inside. This betrays my own arrogance I suppose, and I’ve no doubt I’ve missed a few profound insights this way. But given the choice I prefer to read about things I don’t know, books that don’t promise to back up my existing ideas. Then there are those like Gladwell’s Blink or The Tipping Point, books whose title more or less says it all. A quick rifle through the pages of these books in an airport bookshop – in that peculiar pre-flight mode of having no time and time on your hands – is enough to get the gist, and speculate as to their point.

The Black Swan almost fell into this category, but a recommendation by Paul Schütze and a few others meant that I did pick it up – at Melbourne Airport, ironically – and consumed it voraciously.

It’s not so much a popular science book as a popular statistics
book, not a genre I would’ve thought probable to emerge, and thus
something of a black swan in itself. Black swans are, according to that
back cover blurb, “random events that underlie our lives … their impact
is huge; they’re nearly impossible to predict; yet after they happen we
always try to rationalize them.” So describes the phenomenon that
underpins this book – drawn from the well-known example indicating the flaws in inductive reasoning,
and of the corollary: the validity of falsification – but that doesn’t
describe the significant consequences of Nassim Nicholas Taleb’s
message here.

As a book, it serves as a bracing rejoinder to many ideas I’d begun
to almost instinctively value, particularly the value of expertise and
forecasting. And while Taleb doesn’t go after all expertise (although
he prefers “sophisticated craft”), nor does he recommend not attempting
to predict, he systematically takes apart much of the practice of
predicting, planning and popular approaches to working with data. Put
simply, too much of what is taken to be expert prediction is based on
simplistic, flawed induction and subjective, overly-optimistic, and
simplistic stories, no matter how dressed up in economic theory, and
thus horribly exposed to random black swans.

Taleb forcefully asserts that black swans are the only things that
meaningfully and powerfully shape our world, whether it’s the
stockmarket crash of 1987, September 11th, the spread of the internet
or the assassination of minor Hungarian royalty in Sarajevo in 1914. So
when portfolio managers talk about measuring risk without taking into
account the possibility of black swans – which they do – well, Taleb
asserts this is no better than astrology. And given that astrology
generally gets one page at best in newspapers – one more than it
deserves, of course – yet finance often an entire section, it is thus
more dangerous than astrology. (Incidentally, Taleb is scathing about
the value of daily newspapers in general, recommending a total
rejection of them – “additional knowledge of the minutiae of daily
business can be useless, even actually toxic” – due to our problem with
being swayed by the sensational, and narrative, in the broadest sense).

Specifically, most recent economic modelling has been predicated on
Gaussian distribution, which should only apply in certain
circumstances, and not in scalable situations (such as finance) where
fractal distribution has more to offer.

His primary focus is indeed finance and economics, something I
generally have little more than an academic interest in. Yet I find
much here that made me pause to consider new ways of thinking my own
work through, and equally reinforced that design, in general, should
more deeply understand economic behaviour – or at relevant subsets of
it. Equally, those working with data in almost any field will find much
of value here.

In the field of finance Taleb talks from near unbeatable position of
having been a day trader for many years as well as an educator and
writer, thus working with financial data from the ‘bottom-up’, as he
puts it. His scorn for academics who reject practioners is consistent
throughout, and he much prefers the instinctive approach of the latter,
derived from revelling in the complexity of everyday data.

It’s particularly instructive to be reading it amidst the smoking
ruins of the global financial collapse of 2008, a scenario largely
predicted by Taleb in this book – including a specific footnoted
warning as to the precarious position of Fannie Mae, never mind the
dangerously interconnected nature of the handful of financial players
that dominate the global banking system. (Not that I am truly amidst
those smoking ruins as yet; Australia is one of the very few developed
nations the OECD predicts will avoid recession. But you get my point,
and I’m now extremely aware of the glaring presence of the phrase ‘the
OECD predicts’ in that previous sentence. Indeed the data when I wrote
this piece (04.12.08), two weeks after that OECD prediction, indicated
the Australian economy was now sailing extremely close to ‘negative
growth’. Still, better to be sailing at all, and the relatively
conservative practices of Australian banks and government (plus trade with China-India) has left it,
temporarily at least, as pretty much the only developed 'western'
member of the G20 not in recession.)

Taleb is deeply concerned that much of the analytical framework the
modern world is constructed on is not calibrated to deal with black
swans, the very things that have indeed tended to shape the world, for
better sometimes and yet often for worse.

“That we got here by accident does not mean that we
should continue to take the same risks. We are mature enough a race to
realise this point, enjoy our blessings, and try to preserve, by
becoming more conservative, what we got by luck. We have been playing
Russian roulette; now let’s stop and get a real job.”

Had he written this book a year later, his scathing critique of contemporary financial engineering would hardly have been necessary. He could simply have printed the market data and
listed verbatim quotes from the likes of Alan Greenspan – “the reason I
was shocked (was) because I had been going for 40 years or more with
very considerable evidence that it was working very well”. Or as
Greenspan told Congress, “data put into the risk-management models only
covered the past two decades of euphoria, excluding previous instances
of financial stress” (from ‘Up in smoke’, Australian Literary Review,
December 3 2008). George Soros notes how the crisis has “contradicted
the prevailing theory, which holds that financial markets tend towards
equilibrium”, a Gaussian distribution-based equilibrium which Taleb
proves time and again is an inappropriate concept for capital markets.

Yet fortunately, and despite Taleb’s sometimes merciless
evisceration of his foes, this is a far from nihilistic book and Taleb
replaces much of the rotten economic superstructure with a
sophisticated yet instinctive way of thinking, more suited to taking
advantage of significant unforeseen events.

Equally, do not get overly worried by the presence of the word
‘conservative’ in the quotation above – Taleb’s conservatism applies
mainly to handling of finance and economics, but he appears to
represents a far from conservative kind of person. He does not dismiss
expertise from the position of an anti-intellectual, but instead
provides a vigourous example of perhaps an earlier kind of intellectual
– the broad-minded, highly-cultured and independent intellectual
position long since greatly diminished in number and influence by the
rise of discipline-based experts. Railing against the monocultures
created by ‘nerds’ who focus on one intellectual pursuit to the
eventual exclusion of others, Taleb suggests the importance of
remaining open-minded, curious and interdisciplinary, a position I
can’t endorse enough. Only in his careless and frequent lashing out at
pretty much all social sciences do we find him to be rejecting his own
medicine – it’s an attack that may often be warranted, but certainly
not always.

Yet most experts are systematically taken apart by much detailed
evidence, from the apparent lack of anyone to usefully predict
financial markets ever, to the detailed work of psychologist
Philip Tetlock, who conducted psychological research on ‘experts’ with
quite astonishing results.

Taleb frequently attacks the egotism of these ‘experts’, particularly those who loudly proclaim the virtues of their (inherently flawed)
methodologies, accepting the rewards that a gullible world endows upon
them. (And not just the ‘fuck you’ money, as Taleb has it; he is
particularly scathing about the Nobel prize for economics). Yet Taleb’s
own ego appears to be in a fairly healthy condition throughout, thanks
to the book often possessing the air of a subtle kind of autobiography
as much as an intellectual treatise. This makes the book a far more
interesting read, of course, and The Black Swan is often
amusing and always engaging, even when knee-deep in theory. Taleb comes
across as complex, flawed, impassioned, wise, and sounds like an
extremely good person to share a bottle of wine with.

For those were told “there would be no math”, as Chevy Chase’s
befuddled Gerald Ford character would’ve said, you’ve been misinformed.
There is the occasional dose of mildly mind-bending maths in here,
hovering between philosophy and raw statistics. But Taleb easily
signposts ways around this, and is a good-natured guide through it for
those with the stomach. I confess to being lost on a few occasions, but
Taleb ensures that you never lose sight of the central arguments – the
reasons why he’s explaining how casinos or capital markets work in
detail, for instance (at least with casinos, you can calculate the
odds, he notes.)

Partly thanks to the breadth of Taleb’s erudition, the more
interesting passages here concern our psychological makeup, with
advantages and disadvantages endowed by the process of evolution as
much as anything. You’ll find much here to aid understanding of how
people think – and how people act on instinct, for that matter. Beyond
that, our almost subconscious attraction to finding confirmation of
one’s notions through active construction of data to support that
notion is particularly interesting, and worrying.

Taleb recognises
that the open-minded empirical skepticism he recommends takes supreme
efforts, but this is his suggested method – to live as a practitioner
amidst empirical data, and attempt to build knowledge through
falsification rather than confirmation. His advice is rather more
complicated and multi-faceted than this, yet he puts much of this so
elegantly at times that it often appears to be common sense. However,
when you want a detailed summation of his specific advice it is not
presented as punchily as his earlier critique. (I’d also like to know
more about his simulations, for instance).

This ultimately means you have to construct your own findings, which
is no bad thing but does lend a little credence to the idea that Taleb
has intellectually taken apart much of the financial system (just a
year or so before it self-immolated anyway) yet doesn’t produce an
equivalently detailed replacement.

A further criticism of The Black Swan might be that it makes its
points early, and well, but then drives deeper and deeper into
exploring the issue, rather than devoting equivalent weight to possible
solutions. It’s thoroughly fascinating but similar learning can be
found in an excellent essay on behavioural economics and optimism, by the consistently excellent Gideon Haigh in the consistently excellent Australian magazine The Monthly, and in a sharp critique of the US economy post-Greenspan by Joseph A. Stieglitz in Vanity Fair. Stieglitz sums things up by relating Greenspan’s now infamous recent performance:

“The
truth is most of the individual mistakes boil down to just one: a
belief that markets are self-adjusting and that the role of government
should be minimal. Looking back at that belief during hearings this
fall on Capitol Hill, Alan Greenspan said out loud, “I have found a
flaw.” Congressman Henry Waxman pushed him, responding, “In other
words, you found that your view of the world, your ideology, was not
right; it was not working.” “Absolutely, precisely,” Greenspan said.
The embrace by America—and much of the rest of the world—of this flawed
economic philosophy made it inevitable that we would eventually arrive
at the place we are today.” ['Capitalist Fools', Joseph A. Stieglitz, Vanity Fair]

Taleb
draws frequently on Mandelbrot and Popper above all, and with Hayek in
a supporting role. The latter two might lead some to question the
political backdrop to this stance, yet Taleb rarely directly wanders
off towards a fully libertarian direction thankfully. Equally, he rails
against large corporations as much as government. He would find flaws
in laissez-faire solutions, which would continue to produce the
inequalities that his frequently proclaimed humanism rejects.

Those of us who have struggled through nearly 20 years of hearing
that the market was the best way to run everything – from schools and
hospitals to childcare, housing, energy generation and transport
infrastructure – have this recent ‘performance’ of the financial system
itself as an additional indication that, well, it patently isn’t that
simple. It’ll take some time for those purveyors of that mindless
rhetoric to take on board that ever freer markets are not the only
solution – much of this book would quickly help with that
understanding, even if it is written from the perspective of someone
who lives in, and loves at some level, those markets.

Yet now that that same rhetoric has left government often shrivelled
in significance – but not size, oddly – and with its skillset blunted
accordingly, we are not left with much either way. For instance, here
in Sydney, you can’t walk down the streets without bumping into legions
of fiercely bright financial analysts (mainly because they’re looking
at their BlackBerrys). Yet if they could focus their intellect on
something broader than bolstering a system based on financial
self-interest – which behavioural economics indicates is far from
rational and, yes, doesn’t tend towards equilibrium – we might get
somewhere. Their lack of work might even generate a useful 'cognitive surplus', to appropriate Clay Shirky's phrase. Yet Taleb doesn’t really give us any way out of this.
Perhaps as this is a deeper cultural problem, rather than economic.
That Taleb doesn’t go anywhere near that greatest market failure of all
– global warming – is also interesting, but also worrying.

Still, as a bracing, intellectually stimulating, often funny, clear-sighted and conservatively radical read, The Black Swan does begin to back us out of a cul-de-sac. Where we go next is an interesting dilemma.

Below, a few bullet-points indicating particular aspects drawn from The Black Swan
as regards cities, design, data, and so on. It’s particularly sobering
reading when one works in a team with ‘planning’ in the title (urban in
my case) but I can recommend reading alongside the excellent First Read This: Systems Engineering in Practice by Ed van Hinte and Michel Van Tooren (010 Publishers),
which makes some similar cautionary remarks to Taleb, but in the
context of something akin to planning of complex systems nonetheless.
Despite the title, you don’t have to read it first but you do have to
read as well:

———-

The value of not constructing a narrative from data. During the forum ending our recent course at UTS (“The Street as Platform”),
we had a debate as to the value of displaying the data versus
constructing narratives around the data. Whilst the act of selecting
and displaying data is of course its own narrative, many of the
visualisations the students had built were open-ended in nature, in
part informed by Mitchell Whitelaw’s interest in this as an artistic strategy, and mine as a design tool-making strategy. Andrew vande Moere
indicated that he would’ve liked to have seen more narratives drawn out
of the data, in order to illustrate points, or draw the viewer towards
the data, a view several others at the forum agreed with. Mitchell and
I remained uneasy about the desire to pin the data to a narrative
however, and Taleb also takes this position several times, but also
indicates how hard it is.

“Counter to what everyone
believes, not theorizing is an act – thus theorizing can correspond ot
the absence of willed activity, the ‘default’ option. It takes
considerable effort to see facts (and remember them) while withholding
judgment and resisting explanations. And this theorizing disease is
rarely under our control: it is largely anatomical, part of our
biology, so fighting it requires fighting one’s own self. So the
ancient skeptics’ precepts to withhold judgment go against our nature.”

He
is indicating the value of remaining open – but also that
interpretation is so instinctive it is rarely even perceptible by those
doing it. In terms of our data experiment the other day, I think Andrew
may well have been right in most instances there, but that perhaps a
more sophisticated version of our visualisations would’ve resisted
narrative and interpretation even more.

———-

Related to the above:

“The more information you
give someone, the more hypotheses they will formulate along the way,
and the worse off they will be. They see more random noise and mistake
it for information. The problem is that our ideas are sticky: once we
produce a theory, we are not likely to change our minds – so those who
delay developing their theories are better off.”

———-

Also related to the above:

“The world,
epistemologically, is literally a different place to a bottom-up
empiricist. We don’t have the luxury of sitting down to read the
equation that governs the universe; we just observe data and make an
assumption about what the real process might be, and ‘calibrate’ by
adjusting our equation in accordance with additional information. As
events present themselves to us, we compare what we see to what we
expected to see. It is usually a humbling process, particularly for
someone aware of the narrative fallacy, to discover that history runs
forward, not backward.”

———-

On why Benoit Mandelbrot took an interest in finance:

“Mandelbrot’s
answer was, ‘Data, a gold mine of data.’ Indeed, everyone forgets that
he started in economics before moving on to physics and the geometry of
nature. Working with such abundant data humbles us; it provides the the
intuition of the following error: travelling the road between
representation and reality in the wrong direction.”

———-

He writes specifically of the hubris of those who survive a process
– and how the survivor is ill-equipped to commentate from their
position (being largely unaware of the ‘cemetery’, as he puts it, of
non-survivors. The ‘silent evidence’.) Specifically in terms of cities,
and even more specifically, New York City. Or rather the idea of “the
resilience of New York City”, as if it were an internal quality of the
city.

“Let’s put a collection of cities in a
simulator of history: Rome, Athens, Carthage, Byzantium, Tyre, Catal
Hyuk (located in modern-day Turkey, it is one of the first known human
settlements), Jericho, Peoria, and, of course, New York City. Some
cities will survive the harsh conditions of the simulator. As to
others, we know that history might not be too kind. I am sure that
Carthage, Tyre, and Jericho had their local, no less eloquent
(commentator) saying “Our enemies have tried to destroy us many times,
but we always come back more resilient than before. We are now
invincible.”

 

This bias causes the survivor to be an unqualified witness of the
process. Unsettling? The fact that you survived is a condition that may
weaken your interpretation of the properties of the survival, including
the shallow notion of ‘cause’.”

———-

The limits of (some) predictive models:

“Statisticians
have concentrated their efforts in building more sophisticated models
without regard to the ability of such models to more accurately predict
real-life data.”

———-

Some specific advice, which reinforces the urban advantage:

“Collect
as many free nonlottery tickets (those with open-ended payoffs) as you
can, and, once they start paying off, do not discard them. Work hard,
not in grunt work, but in chasing such opportunities and maximising
exposure to them. This makes living in big cities invaluable because
you increase the odds of serendipity. The idea of settling in a rural
area on grounds that one has good communications “in the age of the
internet” tunnels out of such sources of positive uncertainty.”

———-

And Caravaggio’s 'The Fortune-Teller' is a great image to use when opening presentations on foresight and innovation work.

The Black Swan, Nassim Nicholas Taleb (2007) [Amazon US|UK]
Nassim Nicholas Taleb's homepage