Showing posts with label complex systems. Show all posts
Showing posts with label complex systems. Show all posts
27 October 2007
The Second Law of Organization
Doyne Farmer, "The Second Law of Organization", a chapter in The Third Culture: Beyond the Scientific Revolution. (It looks to me like you can read the whole book online, but if you like having a paper book in your hands, you can buy it from Amazon.
Farmer was trained as a physicist; he's been referred to as one of the "pioneers of chaos theory" and has also done work in finance.
He writes, about his disappointment at what he learned in physics classes in college:
This is a more dignified way of saying that "shit happens", which is something that people who are planning things forget. It's why large projects always seem to be late and over budget. The planners think they've factored in everything with, say, more than a 5% chance of happening. (In the car analogy, when setting out for their trip they've thought that there might be heavy traffic, or perhaps if it's raining they've considered the possibility of flooding on some flood-prone road.) But there are a zillion things that could happen, and you can't factor all of those in individually. But if there are enough of those rare things, the probability of one occuring will be close to one, or the expected number of them will be quite large. (Nassim Taleb has written about this in the context of financial markets.) Unfortunately, when you're writing the budget you can't put a line for "shit that will happen". This is actually the same idea that I was previously talking about; large systems (like all the cars in a given city) will have fairly stable properties, even when individual cars (that one that just drove by outside my window) don't.
(By the way, the "Second Law" that Farmer is referring to is a reference to the Second Law of Thermodynamics; he's basically saying that order appears out of chaos often enough that this apparent contradiction of the Second Law of Thermodynamics is itself a law.)
Farmer was trained as a physicist; he's been referred to as one of the "pioneers of chaos theory" and has also done work in finance.
He writes, about his disappointment at what he learned in physics classes in college:
"And most of physics was still focused on pushing and pulling, on the material properties of the universe rather than on its informational properties. By informational properties, I mean those that relate to order and disorder."I have the sense that this is still true, although I'm not a physicist; certainly it is true of the physics classes that my physics-major friends took. And there seems to be an analogy in mathematics -- it's becoming more and more possible to predict things about large random structures. (For example, one can say quite a bit about what a random partition of a large integer, or a random set, or any number of other large random structures looks like; in those cases where there aren't proofs, one can still do simulations and see that "yes, they always look the same", at least if you squint.) It seems to be mostly the people with some CS background that are interested in this, because these are the sort of questions that arise naturally in the analysis of algorithms. But there really do seem to be two different ways to look at a lot of these objects -- you can look at individual trees, but what's the point, since you're probably never going to see any particular tree with 1000 leaves. Or you can look at the class of trees... a beautiful object on its own.
"Many of us find this view [that that everything depends on a set of disconnected "cosmic accidents."] implausible. Why would so many different types of order occur? Why would our situation be so special? It seems more plausible to assume that "accidents tend to happen." An individual automobile wreck may seem like a fluke- -in fact, most automobile wrecks may seem like flukes — but on average they add up. "
This is a more dignified way of saying that "shit happens", which is something that people who are planning things forget. It's why large projects always seem to be late and over budget. The planners think they've factored in everything with, say, more than a 5% chance of happening. (In the car analogy, when setting out for their trip they've thought that there might be heavy traffic, or perhaps if it's raining they've considered the possibility of flooding on some flood-prone road.) But there are a zillion things that could happen, and you can't factor all of those in individually. But if there are enough of those rare things, the probability of one occuring will be close to one, or the expected number of them will be quite large. (Nassim Taleb has written about this in the context of financial markets.) Unfortunately, when you're writing the budget you can't put a line for "shit that will happen". This is actually the same idea that I was previously talking about; large systems (like all the cars in a given city) will have fairly stable properties, even when individual cars (that one that just drove by outside my window) don't.
(By the way, the "Second Law" that Farmer is referring to is a reference to the Second Law of Thermodynamics; he's basically saying that order appears out of chaos often enough that this apparent contradiction of the Second Law of Thermodynamics is itself a law.)
05 July 2007
swarm theory
National Geographic on Swarm Theory -- the field variously known as "complex systems", "emergent behavior", etc. In some sense certain complex systems are greater than the sum of their parts -- an ant colony seems to have some sort of "intelligence", even though a single ant doesn't. Each individual person betting on a horse race has their biases, but in parimutuel betting the final odds (which depend on how many people bet on each horse) very nearly reflect what actually happens. Juries are smarter than each of their individual members -- although only when the jurors act independently. (This makes me wonder: are people better or worse at following complex arguments than they were a couple hundred years ago?)
Public transportation systems, especially in the U.S., are notoriously inefficient; I suspect a large part of this is because they run on the same routes they've historically run on. I have to wonder if this sort of thing could be applied to redesign of public transit; the article mentions other logistical applications. Southwest Airlines, for example, has used this for scheduling. And American Air Liquide, which delivers gases made at a large number of plants to a large number of customers, has redone a lot of their delivery this way.
Unfortunately, though, Air Liquide has found that some of their drivers have to do unintuitive things -- they're no longer delivering from the plant nearest the customer to the customer. I have a feeling this wouldn't work well with people; you can't tell someone "well, it's going to take you an extra half-hour to get to work today, but that saves fifty other people one minute!" even though it does reduce total transit time.
I suspect there might be applications to, say, the dispatching of cabs, or the placing of shared cars (disclaimer: I am not a member of Philly Car Share but lots of people I know are). Or to a hypothetical company like the one Scott Adams suggested which would get people rides by having them text-message their destination to some central server and then hook them up with someone who's willing to give them a ride. Although the appropriate metaphor in this situation is actually more like traditional, decentralized hitchhiking. Hey, maybe that's the solution to the energy crisis!
Public transportation systems, especially in the U.S., are notoriously inefficient; I suspect a large part of this is because they run on the same routes they've historically run on. I have to wonder if this sort of thing could be applied to redesign of public transit; the article mentions other logistical applications. Southwest Airlines, for example, has used this for scheduling. And American Air Liquide, which delivers gases made at a large number of plants to a large number of customers, has redone a lot of their delivery this way.
Unfortunately, though, Air Liquide has found that some of their drivers have to do unintuitive things -- they're no longer delivering from the plant nearest the customer to the customer. I have a feeling this wouldn't work well with people; you can't tell someone "well, it's going to take you an extra half-hour to get to work today, but that saves fifty other people one minute!" even though it does reduce total transit time.
I suspect there might be applications to, say, the dispatching of cabs, or the placing of shared cars (disclaimer: I am not a member of Philly Car Share but lots of people I know are). Or to a hypothetical company like the one Scott Adams suggested which would get people rides by having them text-message their destination to some central server and then hook them up with someone who's willing to give them a ride. Although the appropriate metaphor in this situation is actually more like traditional, decentralized hitchhiking. Hey, maybe that's the solution to the energy crisis!
Subscribe to:
Comments (Atom)