The sub-prime mortgage debacle is a problem of cognitive complexity. A lack of understanding of the risks entailed by deeply nested loan relationships is leading to a lack of trust in the markets, and this uncertainty is leading to a sell-off. More transparency will help – but has its limits.
A story on NPR quotes Lars Christensen, an economist at Danske Bank, as saying that there is no trust in market because of the unknown complexity of the transactions involved. (Adam Davidson, “U.S. Mortgage Market Woes Spread to Europe,” All Things Considered, Aug 10th, 2007; more was broadcast than seems to be in the online version.)
This is a ‘hard intangibles’ problem: intricate chains and bundles of debt arise because there’s no physical limit on the number of times these abstractions can be recomposed and layered, with banks lending to other banks based bundles of bundled loans as collateral. When questions arise about the solvency of one of the root borrowers, it’s in large part because there’s no transparency into what they’re holding. According to the Economist (“Crunch time” Aug 9th 2007), complex, off-balance sheet financial instruments were the catalyst for the market sell-off. Phrases like “investors have begun to worry about where else such problems are likely to crop up” suggest that lack of understanding is driving uncertainty. The entire market is frozen in place, like soldiers in a mine field: one bomb has gone off, but no-one knows where the next is buried.
One of the drivers of the problem, according to the Financial Times (Paul J Davies, “Who is next to catch subprime flu?” Aug 9th 2007), is that low interest rates have propelled investors into riskier and more complex securities that pay a higher yield. “Complexity” is a way of saying that few if any analysts truly understand the inter-relationships among these instruments. The market is facilitated by the use of sophisticated models (i.e. computer programs) that predict the probabilities of default among borrowers, given the convoluted structure of asset-backed bonds. As the crisis has evolved, banks have come to realize – again, suggesting that this was not immediately obvious – that they’re exposed to all forms of credit markets, to more forms of credit risk than they thought.
This suggests a policy response to a world of hard intangibles: enforced transparency. The US stock market, the most advanced in the world, has of necessity evolved to be more transparent in terms of disclosure requirements on company operations than its imitators. According the FT, the Bundesbank is telling all the German institutions to put everything related to sub-prime problems on the table – indicating that increased visibility for the market will improve matters.
It’s striking how little the banks seem to know. BusinessWeek quotes an economist at CalTech as recommending that the Federal Reserve insists firms rapidly evaluate their portfolios to determine exactly how much of the toxic, risky investments they hold – by implication, they don’t know. (Ben Steverman, “Markets: Keeping the Bears at Bay” Aug 10th 2007.) The situation summed up well here:
“The problem here is that the financial industry has created a raft of new, so-called innovative debt products that are hard, even in the best of times, to place an accurate value on. "You don't have the transparency that exists with exchange products," the second-by-second adjustment in a stock price, for example, says Brad Bailey, a senior analyst at the Aite Group. The products are so complex that many investors might have bought them without realizing how risky they are, he says.”
Transparency may be a useful solvent for governance problems in all complex situations. For example, in an unpublished draft paper on network neutrality, analysts at RAND Europe recommend that access to content on next-generation networks be primarily enforced via reporting requirements on network operators, e.g. requiring service providers to inform consumers about the choices they are making when they sign up for a service – one of the keystones of the Internet Freedoms advocated by the FCC under Michael Powell. This may be one of the only ways to provide some degree of management of the modularized value mesh of today’s communications services.
However, transparency as a solution is limited by the degree to which humans can make sense of the information that is made available. If a structure is more complex than we can grasp, then there are limits to the benefits of knowing its details. A hesitate to draw the corollary: that limits should be imposed on the complexity of the intangible structures we create.
Saturday, August 11, 2007
Friday, August 10, 2007
Wang Wei
Wang Wei is one of China’s greatest poets (and painters), the creator of small, evocative landscape poems that are steeped in tranquility and sadness. More than that, though: while the poems are about solitude rooted in a serious practice of ch’an meditation, Wang worked diligently as a senior civil servant all his life.
I found his work via review of Jane Hirshfield’s wonderful collection After. David Hinton’s selection of Wang Wei poems is beautifully wrought. The book is carefully designed, and the Introduction and Notes are very helpful.
Wang’s poetry gives no hint of the daily bureaucracy that he must have dealt with. It’s anchored in his hermitage in the Whole-South Mountain, which was a few hours from the capital city where he worked. Wang was born in the early Tang dynasty to one of the leading families. He had a successful civil service career, passing the entrance exam at the young age of 21 and eventually serving as chancellor. His wife died when he was 29, at which point he established a monastery on his estate. He died at the age of 60.
Deer Park is one of his most famous poems. Here’s Hinton’s translation:
Wang’s work shows that one can make spiritual progress while also participating in society. One does not have to give up the daily life and become a monk, though it’s surely important that one’s mundane activities make a contribution to the good of society.
I found his work via review of Jane Hirshfield’s wonderful collection After. David Hinton’s selection of Wang Wei poems is beautifully wrought. The book is carefully designed, and the Introduction and Notes are very helpful.
Wang’s poetry gives no hint of the daily bureaucracy that he must have dealt with. It’s anchored in his hermitage in the Whole-South Mountain, which was a few hours from the capital city where he worked. Wang was born in the early Tang dynasty to one of the leading families. He had a successful civil service career, passing the entrance exam at the young age of 21 and eventually serving as chancellor. His wife died when he was 29, at which point he established a monastery on his estate. He died at the age of 60.
Deer Park is one of his most famous poems. Here’s Hinton’s translation:
No one seen. Among empty mountains,(For more translations, see here and here.)
hints of drifting voice, faint, no more.
Entering these deep woods, late sunlight
flares on green moss again, and rises.
Wang’s work shows that one can make spiritual progress while also participating in society. One does not have to give up the daily life and become a monk, though it’s surely important that one’s mundane activities make a contribution to the good of society.
Thursday, August 09, 2007
Factoid: 19 million programmers by 2010
According to Evans Data Corp, the global developer population will approach 19 million in 2010. (I found this via ZDNet's ITFacts blog; the EDC site requires registration to even see the press release.) That's quite a big number - the total population of Australia, for example.
Programming will not be a marginal activity, and any fundamental cognitive constraints on our ability to develop increasingly complex problems will be impossible to avoid.
A lot of the growth will come from new countries bringing programmers online: EDC forecasts that the developer population in APAC will grow by 83% increase from 2006 to 2010, compared to just a 15% increase in North America for the same period. This will keep the skill level high, since only very talented people will enter the population, rather than expanding the percentage of the programming population - and thus reducing average skill - in a given country.
Therefore, the qualititative problems of programming won't change much in the next 5-10 years. However, beyond that we may also face the issue of reducing innate skill levels of programmers.
Programming will not be a marginal activity, and any fundamental cognitive constraints on our ability to develop increasingly complex problems will be impossible to avoid.
A lot of the growth will come from new countries bringing programmers online: EDC forecasts that the developer population in APAC will grow by 83% increase from 2006 to 2010, compared to just a 15% increase in North America for the same period. This will keep the skill level high, since only very talented people will enter the population, rather than expanding the percentage of the programming population - and thus reducing average skill - in a given country.
Therefore, the qualititative problems of programming won't change much in the next 5-10 years. However, beyond that we may also face the issue of reducing innate skill levels of programmers.
Sunday, July 22, 2007
IT Project Success: Getting Better, but Big is still Bad
The biennial “Chaos Report” on IT project success from The Standish Group reports that the success/failure ratio flipped between 1994 and 2006. In 1994 the ratio for “flat failures” vs. “complete successes” was a depressing 31% vs. 16%; in 2006 it was a more encouraging 19% vs. 35%. (The work is reported in CIO; the Standish Group web site is remarkably sullen, and doesn’t seem to have any press releases, let alone publicly available recent data.)
On page 2 of the CIO story, the Standish CEO says: “Seventy-three percent of projects with labor cost of less than $750,000 succeed. . . . But only 3 percent of projects a with labor cost of over $10 million succeed. I would venture to say the 3 percent that succeed succeeded because they overestimated their budget, not because they were managed properly.” A $750,000 project is pretty tiny: six developers for six months, at $250,000/developer/year fully loaded. Even a $10 million project is 20 developers for two years.
This result matches received wisdom that large projects are more likely to fail, which I attribute at least in part to the cognitive challenge of wrapping one’s head around large problems.
What should one do about it? It implies that smaller projects are the only way to go – but what if one has ambitious goals? If it’s true that one can construct complex solutions out of many small, simple parts, everything’s fine. But I’m deeply suspicious of the “divide and conquer” or “linearization” assumption. There are many important problems that just can’t be broken up, from inverting a matrix to simulating non-linear systems.
This may be a cultural reality check: many ambitious goals may simply not be achievable. Humility may be the best way to ensure success. I doubt politicians and business executives want to hear this. Trying to fly too high brought Icarus down – exactly as his engineer-father Daedalus had warned.
And things may not get better: as technology progresses, the complexities of our systems will grow, and linear solutions become even less useful. As the interconnectedness and intangibility of society grows, we may have to become more humble, not more bold, because that will be the only way to get stuff done. It’s counter-intuitive that as technology progresses we need to become less, not more, ambitious, but that may be the way things work with the new intangibles.
NOTES
My thanks to Henry Yuen for referring this story.
I have some reservations about the Standish data. It’s proprietary, and there are academics who’ve questioned it for years. CIO provides some background on Chaos Report and its methods in an interview with the CEO; it also summarizes questions about their method. One has to wonder how the sample population has changed over the years. If the number of small projects in the sample has grown over time, then success reported above would increase simply because smaller projects fail less often, not because project management performance has improved.
On page 2 of the CIO story, the Standish CEO says: “Seventy-three percent of projects with labor cost of less than $750,000 succeed. . . . But only 3 percent of projects a with labor cost of over $10 million succeed. I would venture to say the 3 percent that succeed succeeded because they overestimated their budget, not because they were managed properly.” A $750,000 project is pretty tiny: six developers for six months, at $250,000/developer/year fully loaded. Even a $10 million project is 20 developers for two years.
This result matches received wisdom that large projects are more likely to fail, which I attribute at least in part to the cognitive challenge of wrapping one’s head around large problems.
What should one do about it? It implies that smaller projects are the only way to go – but what if one has ambitious goals? If it’s true that one can construct complex solutions out of many small, simple parts, everything’s fine. But I’m deeply suspicious of the “divide and conquer” or “linearization” assumption. There are many important problems that just can’t be broken up, from inverting a matrix to simulating non-linear systems.
This may be a cultural reality check: many ambitious goals may simply not be achievable. Humility may be the best way to ensure success. I doubt politicians and business executives want to hear this. Trying to fly too high brought Icarus down – exactly as his engineer-father Daedalus had warned.
And things may not get better: as technology progresses, the complexities of our systems will grow, and linear solutions become even less useful. As the interconnectedness and intangibility of society grows, we may have to become more humble, not more bold, because that will be the only way to get stuff done. It’s counter-intuitive that as technology progresses we need to become less, not more, ambitious, but that may be the way things work with the new intangibles.
NOTES
My thanks to Henry Yuen for referring this story.
I have some reservations about the Standish data. It’s proprietary, and there are academics who’ve questioned it for years. CIO provides some background on Chaos Report and its methods in an interview with the CEO; it also summarizes questions about their method. One has to wonder how the sample population has changed over the years. If the number of small projects in the sample has grown over time, then success reported above would increase simply because smaller projects fail less often, not because project management performance has improved.
Tuesday, July 17, 2007
Business: a City, not an Ecosystem
Geoffrey West’s work on scaling in cities provides ammunition for my critique of the “business ecosystem” analogy (Ecosystem alert, Eco mumbo jumbo). New Scientist reports on a recent paper by West and co-workers which found that some urbanization processes differ dramatically from biological ones (references below).
Describing the city as an organism is a much-loved metaphor; New Scientist quotes Frank Lloyd Wright waxing lyrical about “thousands of acres of cellular tissue . . . enmeshed by an intricate network of veins and arteries.”
We like to think that cities work like biological entities, just as we like to think that industries work like networks of organisms. But West & Co’s work indicates that the analogy is flawed. As animals get larger, their metabolism slows down. This is true in some respects for cities, but in others the opposite is true. Infrastructure metrics, like the numbers of gas stations and miles of paved roads scale like biological ones: the amount grows less slowly than the size of the city. But for the things that really count, things speed up. For example, measures of wealth creation and innovation - the number of patents, total wages, GDP - grow more rapidly with size. Bigger cities have a faster metabolism than smaller ones, unlike animals.
Industries resemble cities more than they resemble ecosystems. Increasing returns with size and non-zero sums are key characteristics that are found in both cities and industries, but not biological systems. “Business is Urbanism” is a more accurate and productive metaphor than “Business is an Ecosystem”.
P.S. While we’re talking about ecosystems... The very notion of ecosystem is, of course, itself a metaphor: Nature is a System. The American Heritage Dictionary defines a system as “A group of interacting, interrelated, or interdependent elements forming a complex whole.” It is presumed that there is an observable whole; that it can be broken down into elements; and that the elements interact. So when people use the Business is an Ecosystem metaphor, I think what they’re really doing is simply using Business is a System, and cloaking it with the numinous mantle of Nature (cf. Ecosystem alert).
References
Dana Mackenzie, Ideas: the lifeblood of cities, New Scientist, 23 May 2007 (subscription wall)
Bettencourt, Lobo, Helbing, Kuhnert & West, Growth, innovation, scaling, and the pace of life in cities, Proceedings of the National Academy of Sciences, vol 104, p 7301, 24 April 2007
Describing the city as an organism is a much-loved metaphor; New Scientist quotes Frank Lloyd Wright waxing lyrical about “thousands of acres of cellular tissue . . . enmeshed by an intricate network of veins and arteries.”
We like to think that cities work like biological entities, just as we like to think that industries work like networks of organisms. But West & Co’s work indicates that the analogy is flawed. As animals get larger, their metabolism slows down. This is true in some respects for cities, but in others the opposite is true. Infrastructure metrics, like the numbers of gas stations and miles of paved roads scale like biological ones: the amount grows less slowly than the size of the city. But for the things that really count, things speed up. For example, measures of wealth creation and innovation - the number of patents, total wages, GDP - grow more rapidly with size. Bigger cities have a faster metabolism than smaller ones, unlike animals.
Industries resemble cities more than they resemble ecosystems. Increasing returns with size and non-zero sums are key characteristics that are found in both cities and industries, but not biological systems. “Business is Urbanism” is a more accurate and productive metaphor than “Business is an Ecosystem”.
P.S. While we’re talking about ecosystems... The very notion of ecosystem is, of course, itself a metaphor: Nature is a System. The American Heritage Dictionary defines a system as “A group of interacting, interrelated, or interdependent elements forming a complex whole.” It is presumed that there is an observable whole; that it can be broken down into elements; and that the elements interact. So when people use the Business is an Ecosystem metaphor, I think what they’re really doing is simply using Business is a System, and cloaking it with the numinous mantle of Nature (cf. Ecosystem alert).
References
Dana Mackenzie, Ideas: the lifeblood of cities, New Scientist, 23 May 2007 (subscription wall)
Bettencourt, Lobo, Helbing, Kuhnert & West, Growth, innovation, scaling, and the pace of life in cities, Proceedings of the National Academy of Sciences, vol 104, p 7301, 24 April 2007
Sunday, July 15, 2007
When physicists see a power law, they think in terms of phase transitions, and they smell Nobel prizes. They are like sharks with blood in the water
--- Steven Strogatz on the fuss about scaling laws, quoted in New Scientist, Ideas: the lifeblood of cities, 23 May 2007
In context, from New Scientist:
During the 20th century, many researchers studying urban growth focused on economies of scale and their effect on wages. In 1974, Vernon Henderson of Brown University in Rhode Island proposed that cities reach an optimal size by growing until their workers' per capita income reaches a maximum; when it starts to decline, workers leave for other cities. More recently, researchers including West have tried to identify deeper mechanisms behind these societal patterns. Though West is a physicist by training, his reputation stems mostly from his pioneering and controversial work on scaling laws in biology - how things change with size.
What is all the fuss about scaling laws? "Physicists are used to thinking about extremely large systems of identical particles," says Steven Strogatz, a mathematician at Cornell University in Ithaca, New York. Take a piece of iron: at high temperatures, the spins of the particles jiggle around in random directions. If you gradually lower the temperature, the spins stay random until you reach a critical point - then they suddenly line up, and you have a ferromagnet.
This switch from disorder to order is called a phase transition. In the 1960s, physicists noticed that phase transitions follow certain universal patterns, called power laws, even if they have nothing in common physically. Kenneth Wilson of Cornell showed in the 1970s that these power laws come about through the growth of fractal structures, work which won him the Nobel prize in 1982. Since then, Strogatz says, "When physicists see a power law, they think in terms of phase transitions, and they smell Nobel prizes. They are like sharks with blood in the water."
In context, from New Scientist:
During the 20th century, many researchers studying urban growth focused on economies of scale and their effect on wages. In 1974, Vernon Henderson of Brown University in Rhode Island proposed that cities reach an optimal size by growing until their workers' per capita income reaches a maximum; when it starts to decline, workers leave for other cities. More recently, researchers including West have tried to identify deeper mechanisms behind these societal patterns. Though West is a physicist by training, his reputation stems mostly from his pioneering and controversial work on scaling laws in biology - how things change with size.
What is all the fuss about scaling laws? "Physicists are used to thinking about extremely large systems of identical particles," says Steven Strogatz, a mathematician at Cornell University in Ithaca, New York. Take a piece of iron: at high temperatures, the spins of the particles jiggle around in random directions. If you gradually lower the temperature, the spins stay random until you reach a critical point - then they suddenly line up, and you have a ferromagnet.
This switch from disorder to order is called a phase transition. In the 1960s, physicists noticed that phase transitions follow certain universal patterns, called power laws, even if they have nothing in common physically. Kenneth Wilson of Cornell showed in the 1970s that these power laws come about through the growth of fractal structures, work which won him the Nobel prize in 1982. Since then, Strogatz says, "When physicists see a power law, they think in terms of phase transitions, and they smell Nobel prizes. They are like sharks with blood in the water."
Not that weird
Peter Pitsch’s The Innovation Age (1996) made me question something I’ve taken for granted: that complexity and uncertainty in the economy is growing, and doing so at an unprecedented rate. Pitsch’s book is based on this premise, and it made me wonder: what is the evidence?
The number of industry players who are inter-connected may be growing due to the Internet and cheap global travel, but an individual companies is not necessarily directly connected to more counterparts. It’s a bigger graph, but when one looks at individual nodes, the connectivity is much as it has always been.
Uncertainty isn’t new, either. Pitsch mentions the late Middle Ages as a tumultuous period that produced amazing innovation, and the Industrial Revolution was similar. The uncertainty in aggregate may be larger today, but so is the world population; has the normalized per-capita uncertainty grown? A reasonable measure might be stock market volatility. Schwert’s data for the 1859 – 1987 period doesn’t show any trends I can see with the naked eye (G.W. Schwert, “Why Does Stock Market Volatility Change Over Time," Journal of Finance, vol. 44, pp. 1115-1153, 1989). Market uncertainty, at least, is much the same.
I now believe that this is the Special Present Fallacy at play again. We’re always biased to see the present moment as exceptional; the odds are that it’s not.
The number of industry players who are inter-connected may be growing due to the Internet and cheap global travel, but an individual companies is not necessarily directly connected to more counterparts. It’s a bigger graph, but when one looks at individual nodes, the connectivity is much as it has always been.
Uncertainty isn’t new, either. Pitsch mentions the late Middle Ages as a tumultuous period that produced amazing innovation, and the Industrial Revolution was similar. The uncertainty in aggregate may be larger today, but so is the world population; has the normalized per-capita uncertainty grown? A reasonable measure might be stock market volatility. Schwert’s data for the 1859 – 1987 period doesn’t show any trends I can see with the naked eye (G.W. Schwert, “Why Does Stock Market Volatility Change Over Time," Journal of Finance, vol. 44, pp. 1115-1153, 1989). Market uncertainty, at least, is much the same.
I now believe that this is the Special Present Fallacy at play again. We’re always biased to see the present moment as exceptional; the odds are that it’s not.
Thursday, July 12, 2007
Factoids: cost of clinical trials
According to Thomson CenterWatch, a publishing company that focuses on the clinical trials industry, companies need to recruit about 4,000 people to test an experimental drug at a cost of up to $25,000 for each person. That translates into $100 million at the high end.
Source: Advertorial on Clinical Trials in New Scientist, 16 June 2007
Source: Advertorial on Clinical Trials in New Scientist, 16 June 2007
Factoid: A typical cellphone user spends 80% of his or her time communicating with just four other people
--- Source: Stefana Broadbent, an anthropologist who leads the User Adoption Lab at Swisscom, cited by the Economist in Tech Quaterly story on June 9, 2007: Home truths about telecoms.
They also quote her thus: "The most fascinating discovery I've made this year is a flattening in voice communication and an increase in written channels. . . . Users are showing a growing preference for semi-synchronous writing over synchronous voice." The Economist's gloss: "Her research in Switzerland and France found that even when people are given unlimited cheap or free calls, the number and length of calls does not increase significantly. This may be because there is only so much time you can spend talking; and when you are on the phone it is harder to do other things. Written channels such as e-mail, text-messaging and IM, by contrast, are discreet and allow contact to be continuous during the day."
It seems writing really is a useful alternative channel. I guess there's a reason why the Blackberry was so succesful.
They also quote her thus: "The most fascinating discovery I've made this year is a flattening in voice communication and an increase in written channels. . . . Users are showing a growing preference for semi-synchronous writing over synchronous voice." The Economist's gloss: "Her research in Switzerland and France found that even when people are given unlimited cheap or free calls, the number and length of calls does not increase significantly. This may be because there is only so much time you can spend talking; and when you are on the phone it is harder to do other things. Written channels such as e-mail, text-messaging and IM, by contrast, are discreet and allow contact to be continuous during the day."
It seems writing really is a useful alternative channel. I guess there's a reason why the Blackberry was so succesful.
Sunday, July 08, 2007
Ecosystem alert
When you see references to ecosystems in a business story, raise the shields. Someone is trying to mess with your mind.
The current Business Week has two good examples. An adulatory story about the "Apple ecosystem" (Welcome to Planet Apple, which ran as Welcome to Apple World in hard copy) describes how the company has built its network of partners. Implicit is Iansiti and Levien's notion that the most influential companies are "keystone species in an ecosystem." As I argued in Eco mumbo jumbo, the analogy is flawed in a long list of ways. For example: species don't choose to be keystones; companies interact vountarily, but one organism consumes another against its will; and biological systems have neither goals nor external regulators, whereas industries have both.
The ecosystem analogy is used unthinkingly in this story, judging by the hodgpodge of other metaphors that are used: "[the] ecosystem has morphed from a sad little high-tech shtetl into a global empire," "[its] new flock of partners," "a gated, elitist community," "the insular world of the Mac," "the Apple orchard . . . is still no Eden." Note, though, that most of them refer to places, with a nod to nature.
To get a sense of what's really going on when the ecosystem metaphor is used, let's look at another story, Look Who's Fighting Patent Reform. Computing companies have been pushing for patent reform on Capitol Hill, but "[t]he past few weeks have brought an unexpected surge of opposition from what one lobbyist calls the 'innovation ecosystem'—a sprawling network of entrepreneurs, venture capitalists, trade groups, drug and medical equipment manufacturers, engineering societies, and research universities." It's a term used by the special pleader. The only substantive resemblance to an ecosystem is that these groups connect to each other in network. The rhetorical benefit, though, is to invoke the commonplace Nature Is Good. Nature is unspoiled, bountiful, self-regulating: the antithesis of concrete-covered recklessly-regulating partisan politicking. Nature is a metaphor that appeals to both sides of the political divide: it's organic, but competitive; it's an inter-related, but dynamic; it's nurturing, but stern in its consequences. It's therefore ideal when trying to put a halo around an otherwise unsympathetic subject.
The current Business Week has two good examples. An adulatory story about the "Apple ecosystem" (Welcome to Planet Apple, which ran as Welcome to Apple World in hard copy) describes how the company has built its network of partners. Implicit is Iansiti and Levien's notion that the most influential companies are "keystone species in an ecosystem." As I argued in Eco mumbo jumbo, the analogy is flawed in a long list of ways. For example: species don't choose to be keystones; companies interact vountarily, but one organism consumes another against its will; and biological systems have neither goals nor external regulators, whereas industries have both.
The ecosystem analogy is used unthinkingly in this story, judging by the hodgpodge of other metaphors that are used: "[the] ecosystem has morphed from a sad little high-tech shtetl into a global empire," "[its] new flock of partners," "a gated, elitist community," "the insular world of the Mac," "the Apple orchard . . . is still no Eden." Note, though, that most of them refer to places, with a nod to nature.
To get a sense of what's really going on when the ecosystem metaphor is used, let's look at another story, Look Who's Fighting Patent Reform. Computing companies have been pushing for patent reform on Capitol Hill, but "[t]he past few weeks have brought an unexpected surge of opposition from what one lobbyist calls the 'innovation ecosystem'—a sprawling network of entrepreneurs, venture capitalists, trade groups, drug and medical equipment manufacturers, engineering societies, and research universities." It's a term used by the special pleader. The only substantive resemblance to an ecosystem is that these groups connect to each other in network. The rhetorical benefit, though, is to invoke the commonplace Nature Is Good. Nature is unspoiled, bountiful, self-regulating: the antithesis of concrete-covered recklessly-regulating partisan politicking. Nature is a metaphor that appeals to both sides of the political divide: it's organic, but competitive; it's an inter-related, but dynamic; it's nurturing, but stern in its consequences. It's therefore ideal when trying to put a halo around an otherwise unsympathetic subject.
Friday, July 06, 2007
Trading on News
Follow the money, if you want to know where the action is in AI (and most other things). Trading houses are buying tagged news feeds so that they can process them as input for algorithmic trading. That'sll have to be a pretty smart news reader.
The Economist story that reports this development contains these factoids:
Taken a step further, I can imagine the trading houses selling re-tagged news feeds back to Dow Jones and Reuters. Suitably anonimized, aggregated, and delayed to protect the individual movers, information on which news items triggered trades would be useful at second order. And then the news providers can sell the re-re-tagged feeds to the traders, who'll then sell back the re-re-re-tagged . . .
The Economist story that reports this development contains these factoids:
- Algorithmic trading accounts for a third of all share trades in America
- The Aite Group reckons it will make up more than half the share volumes and a fifth of options trades by 2010
- The new London Stock Exchange system catering to the growth of algorithmic trading cuts trading times down to ten milliseconds; on its first day, it processed up to 1,500 orders a second, compared with 600 using its previous system
Taken a step further, I can imagine the trading houses selling re-tagged news feeds back to Dow Jones and Reuters. Suitably anonimized, aggregated, and delayed to protect the individual movers, information on which news items triggered trades would be useful at second order. And then the news providers can sell the re-re-tagged feeds to the traders, who'll then sell back the re-re-re-tagged . . .
Tuesday, July 03, 2007
Factoid: Americans spent only 0.2% of their money but 10% of their time on the internet
Source: Austan Goolsbee and Peter Klewnow, "Valuing Consumer Products by the Time Spent Using Them: An Application to the Internet," draft available at http://faculty.chicagogsb.edu/austan.goolsbee/research/timeuse.pdf. This result suggests that conventional consumer surplus calculations significantly understate the value of the internet.
Paper abstract:
Paper abstract:
For some goods, the main cost of buying the product is not the price but rather the time it takes to use them. Only about 0.2% of consumer spending in the U.S., for example, went for Internet access in 2004 yet time use data indicates that people spent around 10% of their entire leisure time going online. For goods like that, estimating price elasticities with expenditure data can be difficult and, therefore, estimated welfare gains highly uncertain. We show that for time-intensive goods like the Internet, a simple model in which both expenditure and time contribute to consumption can be used to estimate the consumer gains to a good using just the data on time use and the opportunity cost of people's time (i.e., the wage). The theory predicts that higher wage internet subscribers should spend less time online (for non-work reasons) and the degree to which that is true determines the elasticity of demand. Based on expenditure and time use data and our elasticity estimate, we calculate that consumer surplus from the Internet may be around 2% of full-income, or several thousand dollars. This is an order of magnitude larger than what one obtains from a back-of-the-envelope calculation using data from expenditures.
Monday, July 02, 2007
The Known, the Unknown, the Unknowable
Thanks to a acknowledgement in Nassim Taleb's The Black Swan, I've found this fascinating program on The Known, Unknown, and Unknowable run by Jesse Ausubel of the Sloan Foundation.
Gomory outlines this vision in a short essay, published in Scientific American in 1995. He believes that the artificial is simpler, and thus more predictable, than the natural. However, he notes: "Large pieces of software, as they are expanded and amended, can develop a degree of complexity reminiscent of natural objects, and they can and do behave in disturbing and unpredictable ways."
The program led to a workshop at Columbia in 2000; I look forward to digging in to the proceedings. One of the papers that caught my eye was Ecosystems and the Biosphere as Complex Adaptive Systems by Simon Levin. It seems an open question whether evolution increases the resiliency of ecosystems or leads to criticality - a very important matter for business people hoping that aping ecosystems will improve stability!
Gomory outlines this vision in a short essay, published in Scientific American in 1995. He believes that the artificial is simpler, and thus more predictable, than the natural. However, he notes: "Large pieces of software, as they are expanded and amended, can develop a degree of complexity reminiscent of natural objects, and they can and do behave in disturbing and unpredictable ways."
The program led to a workshop at Columbia in 2000; I look forward to digging in to the proceedings. One of the papers that caught my eye was Ecosystems and the Biosphere as Complex Adaptive Systems by Simon Levin. It seems an open question whether evolution increases the resiliency of ecosystems or leads to criticality - a very important matter for business people hoping that aping ecosystems will improve stability!
Thursday, June 28, 2007
In the last fifty years, the ten most extreme days in the financial markets represent half the returns.
Factoid (verbatim) from Nassim Nicholas Taleb's The Black Swan: The Impact of the Highly Improbable, Random House 2007, p. 275.
In the caption to Figure 14 on the following page, which illustrates S&P 500 returns, Taleb observes: "This is only one of many such tests. While it is quite convincing on a casual read, there are many more-convincing ones from a mathematical standpoint, such as the incidence of 10 sigma events."
(This post marks the end of my attempt to use MSN Spaces for my Factoids site. It was just too clunky. I'll now post factoids here; as they pile up, you'll find all of them by clicking on the "factoids" label at the end of the post.)
In the caption to Figure 14 on the following page, which illustrates S&P 500 returns, Taleb observes: "This is only one of many such tests. While it is quite convincing on a casual read, there are many more-convincing ones from a mathematical standpoint, such as the incidence of 10 sigma events."
(This post marks the end of my attempt to use MSN Spaces for my Factoids site. It was just too clunky. I'll now post factoids here; as they pile up, you'll find all of them by clicking on the "factoids" label at the end of the post.)
Wednesday, June 27, 2007
Eco mumbo jumbo
I’m coming to the conclusion that the “business ecosystem” metaphor is nonsense. That’s a pity, since I speculated in Tweaking the Web Metaphor that the food web might be a useful metaphor for the internet, conceived as a “module ecosystem.” [1] Bugs in the business ecosystem mapping would be even more unfortunate for people who’ve made strategic business decisions on the basis of this flawed metaphor.
“Business is an ecosystem” is an analogy, and like any argument from analogy it is valid to the extent that the essential similarities of the two concepts are greater than the essential differences. I will try to show (at too much length for a blog post, I know...) that the differences are much greater than the similarities.
This biological analogy is very popular. A Google search on "business ecosystem" yielded about 154,000 hits, "software ecosystem" gave 76,000 hits (Microsoft’s in 47,200 of them), and “computing ecosystem” 18,000 hits.
So where’s the problem?
Let me count the ways.
1. A biological ecosystem is analyzed in terms of species, each of which represents thousands or millions of organisms. Business ecosystems are described in terms of firms: just one of each. A food web of species summarizes billions of interactions among interactions; a business web of companies is simply the interaction among the firms studied.
2. Species are connected, primarily, by flows of energy and nutrients. A is eaten by B is eaten by C is eaten by D, etc. Energy is lost as heat at every step. In the business system, a link primarily represents company B buying something from company A. Goods flow from A to B, and money flows back. Both A and B gain, otherwise they wouldn’t’ve entered into the transaction. Therefore, the system isn’t lossy, as it is in a food web. In fact, gains from trade suggest that specialization leading to more interacting firms leads to more value. The links between companies could also stand for co-marketing ventures relationships, technology sharing and licensing agreements, collusion, cross shareholding, etc; however, these have the same non-zero sum characteristics as monetary exchange does.
3. One might sidestep these problems by claiming that species are mapped to firms, and individual animals are mapped to the products that a firm sells. That solves the multiplicity mismatch in #1, and, if one just considers the material content of products, the entropy problem in #2. However, two problems remain. First, the value of products is mostly in the knowledge they embody, not their matter; knowledge (aka value add) is created at every step, the inverse of what happens with the 2nd Law of Thermodynamics. Second, companies sell many diverse products. The fudge only works if a species is mapped to a product unit (in fact, to the part of a product unit that produces a single SKU), rather than to a firm.
4. Species change slowly, and their role in an ecosystem changes very slowly; on the other hand, companies can change their role in the blink of an eye through merger, acquisition or divestiture. Interactions between firms can be changed by contract, whereas that between species is not negotiable except perhaps over very long time scales by evolution of defensive strategies).
5. Biological systems are unstable; the driving force of ecological succession is catastrophe. [2] Businesses seek stability, and the biological metaphor is used as a source of techniques to increase resilience; see e.g. Iansiti and Levien’s claim that keystone species lead to increase stability in an ecosystem. [3], [4] If one seeks stability, biological systems are not a good place to look.
6. Biological systems don’t have goals, but human ones do. There are no regulatory systems external to ecosystems, but many, such as rule of law and anti-trust, in human markets. Natural processes don’t care about equity or justice, but societies do, and impose them on business systems. If ecosystems were a good model for business networks, there would be no need for anti-trust in markets.
7. End-consumers are not represented at all in the “business ecosystem” model. Von Hippel and others [5] who study collaborative innovation could be seen to be pointing to customers - or at least some of them - as a node in the business ecosystem, but the same problems about singularity/multiplicity noted above applies here.
8. Companies are exhorted to invest in their ecosystem if they want to keystone species. Keystone species don’t necessarily (or usually) represent a lot of biomass, so it’s not clear why a firm would want to be a keystone. (And of course, the metaphor leaves unstated whether biomass maps to total revenue, profitability, return on investment, or something else.) More generally: being a keystone species isn’t a matter of choice for the animal concerned; the keystone relationship arises from the interactions among species as a matter of course.
The business ecosystem metaphor in use
Iansiti and Levien are high profile proponents of business ecosystems. [3] [4] In The Keystone Advantage, they motivate the analogy between business networks and biological ecosystems by arguing that both are “formed by large, loosely connected networks of entities”, both “interact with each other in complex ways”, and that “[f]irms and species are therefore simultaneously influenced by their internal capabilities and by their complex interactions with the rest of the ecosystem.” They state that the key analogy they draw “is between the characteristic behavior of a species in an ecosystem and the operating strategy of a strategic business unit.” They declare the stakes when they continue: “To the extent that the comparison of business units to ecosystems [I presume they mean “to species”] is a valid one, it suggests that some of the lessons from biological networks can fruitfully be applied to business networks.”
To caricature their argument: Ecosystems are networks; business networks are networks; therefore business networks are ecosystems. Hmmm...
They were more circumspect in the papers that preceded the book, where they try to dodge the weakness of the analogy that underpins their argument by contending that they don’t mean it: “[W]e are not arguing here that industries are ecosystems or even that it makes sense to organize them as if they were, but that biological ecosystems serve both as a source of vivid and useful terminology as well as a providing some specific and powerful insights into the different roles played by firms” ([4], footnote 10). They want to have it both ways: get the rhetorical boost of a powerful biological metaphor, but avoid dealing with parts of the mapping that are inaccurate or misleading. As they noted in their book: If industries cannot be compared to ecosystems, then their insights cannot be validated by the analogy. However, they do attempt a mapping. For example, they attempt to answer the question “What makes a healthy business ecosystem?” by examining ecosystem phenomena like (1) hubs which are said to account for the fundamental robustness of nature’s webs, (2) robustness measured by survival rates in a given ecosystem, (3) productivity of an ecosystem analogized to total factor productivity analysis in economics, and (4) niche creation.
In most if not all cases, the appeal to ecosystem is very superficial; no substantive analogy is drawn. For example, Messerschmitt and Szyperski’s book [6], which made it into softcover, is entitled Software Ecosystem, but its remit seems to be simply to examine software “in the context of its users, its developers, its buyers and sellers, its operators, society and government, lawyers, and economics”; the word ecosystem doesn’t even appear in the index. (Disclaimer: I haven’t read the book.) The word ecosystem is simply meant to evoke a community of interdependent actors, with no reference beyond that to dynamics or behavior.
A more generic flaw with the business ecosystem metaphor is that most people are more familiar with businesses than with ecosystems. Successful metaphors usually explain complex or less-known things in terms of simpler, more familiar ones. Shall I compare thee to a Summer's day? The rhetorical appeal of the business ecosystem analogy must lie beyond its rather weak ability to make domesticate a strange idea.
Why do careful scholars resort to the ecosystem metaphor in spite of its obvious flaws? The image of nature is so powerful that it is a symbol too potent to pass up. Nature represents The Good (at least in our culture at this time), and therefore an appeal to a natural order is a compelling argument if one’s claims bear some resemblance to what’s happening in nature. However, if nothing else, this reminds me of Hegel’s historicist cop-out that what is real is rational, and what is rational is real. Just because nature is constructed in a certain way doesn’t mean that industries should be.
Perhaps my standards for metaphors are too high. To me, a conceptual metaphor is a mapping one set of ideas to another; one has to take the “bad” elements of the mapping with the “good”. If the good outweighs the bad, and if the metaphor produces insight and new ideas, then the analogy has value. Others just take the “good” and simply ignore the “bad”. For me, a metaphor is a take-it-all-or-leave-it set menu, not something to pick from a la carte.
Tentative conclusion
Does this all matter? Yes, but I still have to work out the details. For now I just claim that the weakness of the mapping between biological and business systems means that any argument that one might make about the goodness of “business ecosystems” in general and “keystone species” in particular is potentially misleading. It could delude firms into make unsound investments, e.g. in “building ecosystems,” and lead policy makers into dangerous judgments.
Notes
[1] The module ecosystem differs from the business ecosystem in that species, the nodes of the food web, are mapped to functional modules, rather than to individual companies. However, the glaring weaknesses of the business ecosystem metaphor undermine my confidence in the whole approach.
[2] John Harte, in “Business as a Living System: The Value of Industrial Ecology A Roundtable Discussion,” California Management Review (Spring 2001), argues that the ecological sustainability practices under the banner of “industrial ecology” are worthy and important, but that they do not mimic the way natural ecosystems work. His ideas are reflected in items #2, #5 and #6. He also notes that human processes are much more efficient in using waste heat than natural ones are – photosynthesis is only about a half a percent efficient, whereas power plants at 30% are sixty times more efficient. Note, however, that Industrial ecology, defined as the proposition that industrial systems should be seen as closed-loop ecosystems where wastes of one process become inputs for another process (wikipedia, ISIE) differs from the business ecosystem idea as I treat it here, i.e. that industry organization (regardless of ecological impact) can be understood as an ecosystem.
[3] Marco Iansiti and Roy Levien, “The New Operational Dynamics of Business Ecosystems: Implications for Policy, Operations and Technology Strategy,” Harvard Business School Working Paper 03-030 (2003)
[4] Marco Iansiti and Roy Levien, The Keystone Advantage: What the New Dynamics of Business Ecosystems Mean for Strategy, Innovation, and Sustainability, Harvard Business School Press, 2004
[5] Eric von Hippel, Democratizing Innovation (2005), and e.g. Charles Leadbeater
[6] David Messerschmitt and Clemens Szyperski, Software Ecosystem – Understanding an Indispensable Technology and Industry, MIT Press, 2003
“Business is an ecosystem” is an analogy, and like any argument from analogy it is valid to the extent that the essential similarities of the two concepts are greater than the essential differences. I will try to show (at too much length for a blog post, I know...) that the differences are much greater than the similarities.
This biological analogy is very popular. A Google search on "business ecosystem" yielded about 154,000 hits, "software ecosystem" gave 76,000 hits (Microsoft’s in 47,200 of them), and “computing ecosystem” 18,000 hits.
So where’s the problem?
Let me count the ways.
1. A biological ecosystem is analyzed in terms of species, each of which represents thousands or millions of organisms. Business ecosystems are described in terms of firms: just one of each. A food web of species summarizes billions of interactions among interactions; a business web of companies is simply the interaction among the firms studied.
2. Species are connected, primarily, by flows of energy and nutrients. A is eaten by B is eaten by C is eaten by D, etc. Energy is lost as heat at every step. In the business system, a link primarily represents company B buying something from company A. Goods flow from A to B, and money flows back. Both A and B gain, otherwise they wouldn’t’ve entered into the transaction. Therefore, the system isn’t lossy, as it is in a food web. In fact, gains from trade suggest that specialization leading to more interacting firms leads to more value. The links between companies could also stand for co-marketing ventures relationships, technology sharing and licensing agreements, collusion, cross shareholding, etc; however, these have the same non-zero sum characteristics as monetary exchange does.
3. One might sidestep these problems by claiming that species are mapped to firms, and individual animals are mapped to the products that a firm sells. That solves the multiplicity mismatch in #1, and, if one just considers the material content of products, the entropy problem in #2. However, two problems remain. First, the value of products is mostly in the knowledge they embody, not their matter; knowledge (aka value add) is created at every step, the inverse of what happens with the 2nd Law of Thermodynamics. Second, companies sell many diverse products. The fudge only works if a species is mapped to a product unit (in fact, to the part of a product unit that produces a single SKU), rather than to a firm.
4. Species change slowly, and their role in an ecosystem changes very slowly; on the other hand, companies can change their role in the blink of an eye through merger, acquisition or divestiture. Interactions between firms can be changed by contract, whereas that between species is not negotiable except perhaps over very long time scales by evolution of defensive strategies).
5. Biological systems are unstable; the driving force of ecological succession is catastrophe. [2] Businesses seek stability, and the biological metaphor is used as a source of techniques to increase resilience; see e.g. Iansiti and Levien’s claim that keystone species lead to increase stability in an ecosystem. [3], [4] If one seeks stability, biological systems are not a good place to look.
6. Biological systems don’t have goals, but human ones do. There are no regulatory systems external to ecosystems, but many, such as rule of law and anti-trust, in human markets. Natural processes don’t care about equity or justice, but societies do, and impose them on business systems. If ecosystems were a good model for business networks, there would be no need for anti-trust in markets.
7. End-consumers are not represented at all in the “business ecosystem” model. Von Hippel and others [5] who study collaborative innovation could be seen to be pointing to customers - or at least some of them - as a node in the business ecosystem, but the same problems about singularity/multiplicity noted above applies here.
8. Companies are exhorted to invest in their ecosystem if they want to keystone species. Keystone species don’t necessarily (or usually) represent a lot of biomass, so it’s not clear why a firm would want to be a keystone. (And of course, the metaphor leaves unstated whether biomass maps to total revenue, profitability, return on investment, or something else.) More generally: being a keystone species isn’t a matter of choice for the animal concerned; the keystone relationship arises from the interactions among species as a matter of course.
The business ecosystem metaphor in use
Iansiti and Levien are high profile proponents of business ecosystems. [3] [4] In The Keystone Advantage, they motivate the analogy between business networks and biological ecosystems by arguing that both are “formed by large, loosely connected networks of entities”, both “interact with each other in complex ways”, and that “[f]irms and species are therefore simultaneously influenced by their internal capabilities and by their complex interactions with the rest of the ecosystem.” They state that the key analogy they draw “is between the characteristic behavior of a species in an ecosystem and the operating strategy of a strategic business unit.” They declare the stakes when they continue: “To the extent that the comparison of business units to ecosystems [I presume they mean “to species”] is a valid one, it suggests that some of the lessons from biological networks can fruitfully be applied to business networks.”
To caricature their argument: Ecosystems are networks; business networks are networks; therefore business networks are ecosystems. Hmmm...
They were more circumspect in the papers that preceded the book, where they try to dodge the weakness of the analogy that underpins their argument by contending that they don’t mean it: “[W]e are not arguing here that industries are ecosystems or even that it makes sense to organize them as if they were, but that biological ecosystems serve both as a source of vivid and useful terminology as well as a providing some specific and powerful insights into the different roles played by firms” ([4], footnote 10). They want to have it both ways: get the rhetorical boost of a powerful biological metaphor, but avoid dealing with parts of the mapping that are inaccurate or misleading. As they noted in their book: If industries cannot be compared to ecosystems, then their insights cannot be validated by the analogy. However, they do attempt a mapping. For example, they attempt to answer the question “What makes a healthy business ecosystem?” by examining ecosystem phenomena like (1) hubs which are said to account for the fundamental robustness of nature’s webs, (2) robustness measured by survival rates in a given ecosystem, (3) productivity of an ecosystem analogized to total factor productivity analysis in economics, and (4) niche creation.
In most if not all cases, the appeal to ecosystem is very superficial; no substantive analogy is drawn. For example, Messerschmitt and Szyperski’s book [6], which made it into softcover, is entitled Software Ecosystem, but its remit seems to be simply to examine software “in the context of its users, its developers, its buyers and sellers, its operators, society and government, lawyers, and economics”; the word ecosystem doesn’t even appear in the index. (Disclaimer: I haven’t read the book.) The word ecosystem is simply meant to evoke a community of interdependent actors, with no reference beyond that to dynamics or behavior.
A more generic flaw with the business ecosystem metaphor is that most people are more familiar with businesses than with ecosystems. Successful metaphors usually explain complex or less-known things in terms of simpler, more familiar ones. Shall I compare thee to a Summer's day? The rhetorical appeal of the business ecosystem analogy must lie beyond its rather weak ability to make domesticate a strange idea.
Why do careful scholars resort to the ecosystem metaphor in spite of its obvious flaws? The image of nature is so powerful that it is a symbol too potent to pass up. Nature represents The Good (at least in our culture at this time), and therefore an appeal to a natural order is a compelling argument if one’s claims bear some resemblance to what’s happening in nature. However, if nothing else, this reminds me of Hegel’s historicist cop-out that what is real is rational, and what is rational is real. Just because nature is constructed in a certain way doesn’t mean that industries should be.
Perhaps my standards for metaphors are too high. To me, a conceptual metaphor is a mapping one set of ideas to another; one has to take the “bad” elements of the mapping with the “good”. If the good outweighs the bad, and if the metaphor produces insight and new ideas, then the analogy has value. Others just take the “good” and simply ignore the “bad”. For me, a metaphor is a take-it-all-or-leave-it set menu, not something to pick from a la carte.
Tentative conclusion
Does this all matter? Yes, but I still have to work out the details. For now I just claim that the weakness of the mapping between biological and business systems means that any argument that one might make about the goodness of “business ecosystems” in general and “keystone species” in particular is potentially misleading. It could delude firms into make unsound investments, e.g. in “building ecosystems,” and lead policy makers into dangerous judgments.
Notes
[1] The module ecosystem differs from the business ecosystem in that species, the nodes of the food web, are mapped to functional modules, rather than to individual companies. However, the glaring weaknesses of the business ecosystem metaphor undermine my confidence in the whole approach.
[2] John Harte, in “Business as a Living System: The Value of Industrial Ecology A Roundtable Discussion,” California Management Review (Spring 2001), argues that the ecological sustainability practices under the banner of “industrial ecology” are worthy and important, but that they do not mimic the way natural ecosystems work. His ideas are reflected in items #2, #5 and #6. He also notes that human processes are much more efficient in using waste heat than natural ones are – photosynthesis is only about a half a percent efficient, whereas power plants at 30% are sixty times more efficient. Note, however, that Industrial ecology, defined as the proposition that industrial systems should be seen as closed-loop ecosystems where wastes of one process become inputs for another process (wikipedia, ISIE) differs from the business ecosystem idea as I treat it here, i.e. that industry organization (regardless of ecological impact) can be understood as an ecosystem.
[3] Marco Iansiti and Roy Levien, “The New Operational Dynamics of Business Ecosystems: Implications for Policy, Operations and Technology Strategy,” Harvard Business School Working Paper 03-030 (2003)
[4] Marco Iansiti and Roy Levien, The Keystone Advantage: What the New Dynamics of Business Ecosystems Mean for Strategy, Innovation, and Sustainability, Harvard Business School Press, 2004
[5] Eric von Hippel, Democratizing Innovation (2005), and e.g. Charles Leadbeater
[6] David Messerschmitt and Clemens Szyperski, Software Ecosystem – Understanding an Indispensable Technology and Industry, MIT Press, 2003
Subscribe to:
Posts (Atom)