Information Processing

[フレーム]

[フレーム]

Highly recommended. Read this article, which will enable you to ignore 99% of mass media and 90% of "expert" commentary on this topic.

THE WAR NERD: TAIWAN — THE THUCYDIDES TRAPPER WHO CRIED WOOF

... The US/NATO command may be woofing just to get more ships and planes funded, but woofing can go badly wrong. The people you’re woofing at may think you really mean it. That’s what came very close to happening in the 1983 Able Archer NATO exercises. The woofing by Reagan and Thatcher in the leadup to those exercises was so convincing to the Soviet woof-ees that even the moribund USSR came close to responding in real—like nuclear—ways.

That’s how contingency plans, domestic political theatrics, and funding scams can feed into each other and lead to real wars.

Military forces develop contingency plans. That’s part of their job. Some of the plans to fight China are crazy, but some are just plausible enough to be worrying, because somebody might start thinking they could work.

... What you do with a place like Xinjiang, if you’re a CIA/DoD planner, is file it under “promote insurgency” — meaning “start as many small fires as possible,” rather than “invade and begin a conventional war.”

And in the meantime, you keep working on the real complaints of the Uyghur and other non-Han ethnic groups, so that if you do need to start a conventional war in the Formosa Straits, you can use the Uyghur as a diversion, a sacrifice, by getting them to rise up and be massacred. Since there’s a big Han-Chinese population in Xinjiang, as the map shows, you can hope to stir up the sort of massacre/counter-massacre whipsaw that leaves evil memories for centuries, leading to a permanent weakening of the Chinese state.

This is a nasty strategy, but it’s a standard imperial practice, low-cost — for the empire, not the local population, of course. It costs those people everything, but empires are not sentimental about such things.

... The Uyghur in Xinjiang would serve the same purpose as the Iraqi Kurds: “straw dogs destined for sacrifice.” If you want to get really cynical, consider that the reprisals they’d face from an enraged Chinese military would be even more useful to the US/NATO side than their doomed insurgency itself.

Atrocity propaganda is very important in 21st c warfare. At the moment, there’s no evidence of real, mass slaughter in Xinjiang, yet we’re already getting propaganda claims about it. Imagine what US/NATO could make out of the bloody aftermath of a doomed insurgency. Well, assuming that US/NATO survived a war with China, a pretty dicey assumption. More likely, CNN, BBC, and NYT would be the first to welcome our new overlords, Kent Brockman style. Those mainstream-media whores aren’t too bright but Lord, they’re agile.

... Xinjiang, by contrast, can easily be imagined as One Giant Concentration Camp. After all, our leading “expert” on the province has never been there, and neither have his readers.

... The era of naval war based on carrier groups is over. They know that, even if they won’t say it.

If there’s a real war with China, the carriers will wait it out in San Diego harbor. I don’t say Honolulu, because even that wouldn’t be safe enough.

I’m not denigrating the courage or dedication of the crews and officers of USN vessels. At any level below JCOS, most of them are believers. But their belief is increasingly besieged and difficult to sustain, like an Episcopalian at Easter. You just can’t think too long about how cheap and effective antiship missiles are and still be a believer in aircraft carriers. As platforms of gunboat diplomacy against weak powers, they’re OK.

... The thing is, and it’s weird you even have to say this: China is a big strong country coming out of an era of deep national humiliation and suffering, proud of its new prosperity. China’s success in lifting a desperately poor population into something like prosperity will likely be the biggest story from this era, when the canonical histories get distilled.

A nation hitting this stage is likely to include a lot of people, especially young men, who are itching to show what their country can do. Their patriotic eagerness is no doubt as gullible as most, but it’s real, and if you pay any attention in the online world, you can’t help seeing it.

People who mouth off about China never seem to imagine that anyone in China might hear, because as we are told over and over again, China-is-an-authoritarian-state. The implication is that nobody in China has any of the nationalistic fervor that we take for granted in our own Anglo states.

... Given the history of US/China relations, from the pogroms against Chinese immigrants to the Chinese Exclusion Act of 1882, through the demonization of Chinese mainlanders in the Cold War (which I remember distinctly from elementary school scare movies), the endless attempts to start insurgencies in Tibet, Xinjiang, and Fujian, to the nonstop violence and abuse of Asians in America, you don’t need to find reasons for Chinese people to want a war.

The odd thing is that most of them don’t seem to. That’s a remarkable testimony to the discipline and good sense of the Chinese public…so far. And it’s also, if you’re thinking clearly, a good reason not to keep provoking China in such gross, pointless ways. A population with that level of discipline and unity, matched with zooming prosperity, technical expertise, and pride on emerging from a long nightmare, is not one to woof at.

Of course the plan in the Pentagon is not real war. The plan is to slow China down, trip it up, “wrong-foot it” as they say in the Commonwealth.

... So what will China do about Taiwan? China could take it right now, if it wanted to pay the price. Everyone knows that, though many fake-news sites have responded with childish, ridiculous gung-ho stories about how “Taiwan Could Win.”

But will China invade? No. Not right now anyway. It doesn’t need to. The Chinese elite has its own constituencies, like all other polities (including “totalitarian” ones), and has to answer to them as circumstances change.

So far China has been extraordinarily patient, a lot more patient than we’d be if China was promising to fight to the death for, say, Long Island. But that can change. Because, as I never tire of repeating, the enemy of the moment has constituencies too. And has to answer to them.

So what happens if the US succeeds in hamstringing China’s economy? Welp, what’s the most reliable distraction a gov’t can find when it wants to unite a hard-pressed population against some distant enemy?

That’s when China might actually do something about Taiwan. ...

See also Strategic Calculus of a Taiwan Invasion. Just a few years ago there were plenty of Western travelers passing through Xinjiang, even by bicycle, vlogging and posting their videos on YouTube. I followed these YouTubers at the time because of my own travel interest in western and south-western China, not for any political reason.

Some analysis of the crap polling and poor election prediction leading up to Nov 2020. See earlier post (and comments): Election 2020: quant analysis of new party registrations vs actual votes, where I wrote (Oct 14)

I think we should ascribe very high uncertainty to polling results in this election, for a number of reasons including the shy Trump voter effect as well as the sampling corrections applied which depend heavily on assumptions about likely turnout. ...

This is an unusual election for a number of reasons so it's quite hard to call the outcome. There's also a good chance the results on election night will be heavily contested.

Eric Kaufmann is Professor of Politics at Birkbeck College, University of London.

UnHerd: ... Far from learning from the mistakes of 2016, the polling industry seemed to have got things worse. Whether conducted by private or public firms, at the national or local, presidential or senatorial, levels, polls were off by wide margins. The Five Thirty-Eight final poll of polls put Biden ahead by 8.4 points, but the actual difference in popular vote is likely to be closer to 3-4 points. In some close state races, the error was even greater.

Why did they get it so wrong? Pollsters typically receive low response rates to calls, which leads them to undercount key demographics. To get around this, they typically weight for key categories like race, education or gender. If they get too few Latinos or whites without degrees, they adjust their numbers to match the actual electorate. But most attitudes vary far more within a group like university graduates, than between graduates and non-graduates. So even if you have the correct share of graduates and non-graduates, you might be selecting the more liberal-minded among them.

For example, in the 2019 American National Election Study pilot survey, education level predicts less than 1% of the variation in whether a white person voted for Trump in 2016. By contrast, their feelings towards illegal immigrants on a 0-100 thermometer predicts over 30% of the variation. Moreover, immigration views pick out Trump from Clinton voters better within the university-educated white population than among high school-educated whites. Unless pollsters weight for attitudes and psychology – which is tricky because these positions can be caused by candidate support – they miss much of the action.

Looking at this election’s errors — which seems to have been concentrated among white college graduates — I wonder if political correctness lies at the heart of the problem.

... According to a Pew survey on October 9, Trump was leading Biden by 21 points among white non-graduates but trailing him by 26 points among white graduates. Likewise, a Politico/ABC poll on October 11 found that ‘Trump leads by 26 points among white voters without four-year college degrees, but Biden holds a 31-point lead with white college graduates.’ The exit polls, however, show that Trump ran even among white college graduates 49-49, and even had an edge among white female graduates of 50-49! This puts pre-election surveys out by a whopping 26-31 points among white graduates. By contrast, among whites without degrees, the actual tilt in the election was 64-35, a 29-point gap, which the polls basically got right.

See also this excellent podcast interview with Kaufmann: Shy Trump Voters And The Blue Wave That Wasn’t Added: ‘Shy Trump Voters’ Re-Emerge as Explanation for Pollsters’ Miss

Bloomberg: ... “Shy Trump voters are only part of the equation. The other part is poll deniers,” said Neil Newhouse, a Republican pollster. “Trump spent the last four years beating the crap out of polls, telling people they were fake, and a big proportion of his supporters just said, ‘I’m not participating.’”

In a survey conducted after Nov. 3, Newhouse found that 19% of people who voted for Trump had kept their support secret from most of their friends. And it’s not that they were on the fence: They gave Trump a 100% approval rating and most said they made up their minds before Labor Day.

Suburbanites, moderates and college-educated voters — especially women — were more likely to report that they had been ostracized or blocked on social media for their support of Trump. ...

... University of Arkansas economist Andy Brownback conducted experiments in 2016 that allowed respondents to hide their support for Trump in a list of statements that could be statistically reconstructed. He found people who lived in counties that voted for Clinton were less likely to explicitly state they agreed with Trump.

“I get a little frustrated with the dismissiveness of social desirability bias among pollsters,” said “I just don’t see a reason you could say this is a total non-issue, especially when one candidate has proven so difficult to poll.”

Dominic Cummings on genomics, healthcare, and innovation in the UK:

Britain could contribute huge value to the world by leveraging existing assets, including scientific talent and how the NHS is structured, to push the frontiers of a rapidly evolving scientific field — genomic prediction — that is revolutionising healthcare in ways that give Britain some natural advantages over Europe and America. We should plan for free universal ‘SNP’ genetic sequencing as part of a shift to genuinely preventive medicine — a shift that will lessen suffering, save money, help British advanced technology companies in genomics and data science/AI, make Britain more attractive for scientists and global investment, and extend human knowledge in a crucial field to the benefit of the whole world.

Those that are interested in the history of science, or in understanding its future, would do well to look at what was being written 10 or so years ago about genomics of complex traits. Whose predictions came true? Whose were dead wrong?

Dominic Cummings: ... Hsu predicted that very large samples of DNA would allow scientists over the next few years to start identifying the actual genes responsible for complex traits, such as diseases and intelligence, and make meaningful predictions about the fate of individuals. Hsu gave estimates of the sample sizes that would be needed. His 2011 talk contains some of these predictions and also provides a physicist’s explanation of ‘what is IQ measuring’. As he said at Google in 2011, the technology is ‘right on the cusp of being able to answer fundamental questions’ and ‘if in ten years we all meet again in this room there’s a very good chance that some of the key questions we’ll know the answers to’. His 2014 paper explains the science in detail. If you spend a little time looking at this, you will know more than 99% of high status economists gabbling on TV about ‘social mobility’ saying things like ‘doing well on IQ tests just proves you can do IQ tests’.

In 2013, the world of Westminster thought this all sounded like science fiction and many MP said I sounded like ‘a mad scientist’. Hsu’s predictions have come true and just five years later this is no longer ‘science fiction’. (Also NB. Hsu’s blog was one of the very few places where you would have seen discussion of CDOs and the 2008 financial crash long BEFORE it happened. I have followed his blog since ~2004 and this from 2005, two years before the crash started, was the first time I read about things like ‘synthetic CDOs’: ‘we have yet another ill-understood casino running, with trillions of dollars in play’. The quant-physics network had much better insight into the dynamics behind the 2008 Crash than high status mainstream economists like Larry Summers responsible for regulation.)

His group and others have applied machine learning to very large genetic samples and built predictors of complex traits. Complex traits like general intelligence and most diseases are ‘polygenic’ — they depend on many genes each of which contributes a little (unlike diseases caused by a single gene).

‘There are now ~20 disease conditions for which we can identify, e.g, the top 1% outliers with 5-10x normal risk for the disease. The papers reporting these results have almost all appeared within the last year or so.’

(One might ask what fraction of PhD economists knew in 2008 what a CDO was or how they were constructed or priced... I was there, and the answer is: very, very few.)

As the deep learning pioneer Jurgen Schmidhuber has emphasized,

... machine learning is itself based on accurate credit assignment. Good learning algorithms assign higher weights to features or signals that correctly predict outcomes, and lower weights to those that are not predictive. His analogy between science itself and machine learning is often lost on critics.

Therefore, to decide how to weight current claims about the future (such as: accurate genomic prediction of many disease risks and complex traits, even including cognitive ability, are right around the corner), one should carefully study the track record of those offering predictions.


Google TechTalk 2011:

[フレーム]


Allen Institute meeting on Genetics of Complex Traits (2018):

[フレーム]

Note Added in response to 2020 Twitter mob attack which attempts to misrepresent my views:

This is not my research. The authors are affiliated with Columbia University and the New York Genome Center.

I do not work on evolutionary history or signals of recent natural selection, but I defend the freedom of other researchers to investigate it.

One has to make a big conceptual leap to claim this research implies group differences. The fact that a certain set of genetic variants has been under selection does not necessarily imply anything about overall differences in phenotype between populations. Nevertheless the work is interesting and sheds some light on natural selection in deep human history.

Racist inferences based on the results of the paper are the fault of the reader, not the authors of the paper or of this blog.




A new paper (94 pages!) investigates signals of recent selection on traits such as height and educational attainment (proxy for cognitive ability). Here's what I wrote about height a few years ago in Genetic group differences in height and recent human evolution:

These recent Nature Genetics papers offer more evidence that group differences in a complex polygenic trait (height), governed by thousands of causal variants, can arise over a relatively short time (~ 10k years) as a result of natural selection (differential response to varying local conditions). One can reach this conclusion well before most of the causal variants have been accounted for, because the frequency differences are found across many variants (natural selection affects all of them). Note the first sentence above contradicts many silly things (drift over selection, genetic uniformity of all human subpopulations due to insufficient time for selection, etc.) asserted by supposed experts on evolution, genetics, human biology, etc. over the last 50+ years. The science of human evolution has progressed remarkably in just the last 5 years, thanks mainly to advances in genomic technology.

Cognitive ability is similar to height in many respects, so this type of analysis should be possible in the near future. ...

The paper below conducts an allele frequency analysis on admixture graphs, which contain information about branching population histories. Thanks to recent studies, they now have enough data to run the analysis on educational attainment as well as height. Among their results: a clear signal that modern East Asians experienced positive selection (~10kya?) for + alleles linked to educational attainment (see left panel of figure above; CHB = Chinese, CEU = Northern Europeans). These variants have also been linked to neural development.

Detecting polygenic adaptation in admixture graphs

Fernando Racimo∗1, Jeremy J. Berg2 and Joseph K. Pickrell1,2 1New York Genome Center, New York, NY 10013, USA 2Department of Biological Sciences, Columbia University, New York, NY 10027, USA June 4, 2017

Abstract
An open question in human evolution is the importance of polygenic adaptation: adaptive changes in the mean of a multifactorial trait due to shifts in allele frequencies across many loci. In recent years, several methods have been developed to detect polygenic adaptation using loci identified in genome-wide association studies (GWAS). Though powerful, these methods suffer from limited interpretability: they can detect which sets of populations have evidence for polygenic adaptation, but are unable to reveal where in the history of multiple populations these processes occurred. To address this, we created a method to detect polygenic adaptation in an admixture graph, which is a representation of the historical divergences and admixture events relating different populations through time. We developed a Markov chain Monte Carlo (MCMC) algorithm to infer branch-specific parameters reflecting the strength of selection in each branch of a graph. Additionally, we developed a set of summary statistics that are fast to compute and can indicate which branches are most likely to have experienced polygenic adaptation. We show via simulations that this method - which we call PhenoGraph - has good power to detect polygenic adaptation, and applied it to human population genomic data from around the world. We also provide evidence that variants associated with several traits, including height, educational attainment, and self-reported unibrow, have been influenced by polygenic adaptation in different human populations.

https://doi.org/10.1101/146043

From the paper:

We find evidence for polygenic adaptation in East Asian populations at variants that have been associated with educational attainment in European GWAS. This result is robust to the choice of data we used (1000 Genomes or Lazaridis et al. (2014) panels). Our modeling framework suggests that selection operated before or early in the process of divergence among East Asian populations - whose earliest separation dates at least as far back as approximately 10 thousand years ago [42, 43, 44, 45] - because the signal is common to different East Asian populations (Han Chinese, Dai Chinese, Japanese, Koreans, etc.). The signal is also robust to GWAS ascertainment (Figure 6), and to our modeling assumptions, as we found a significant difference between East Asian and non- East-Asian populations even when performing a simple binomial sign test (Tables S4, S9, S19 and S24).

Some observations concerning the military balance of power in Asia. Even "experts" I have spoken to over the years seem to be confused about basic realities that are fundamental to strategic considerations.

1. Modern missile and targeting technology make the survivability of surface ships (especially carriers) questionable. Satellites can easily image surface ships and missiles can hit them from over a thousand miles away. Submarines are a much better investment and carriers may be a terrible waste of money, analogous to battleships in the WWII era. (Generals and Admirals typically prepare to fight the previous war, despite the advance of technology, often with disastrous consequences.)

2. US forward bases and surface deployments are hostages to advanced missile capability and would not survive the first days of a serious conventional conflict. This has been widely discussed, at least in some planning circles, since the 1990s. See second figure below and link.

3. PRC could easily block oil shipments to Taiwan or even Japan using Anti-Ship Ballistic Missiles (ASBM) or Anti-Ship Cruise Missiles (ASCM). This is a much preferable strategy to an amphibious attack on Taiwan in response to, e.g., a declaration of independence. A simple threat against oil tankers, or perhaps the demonstration sinking of a single tanker, would be enough to cut off supplies. A response to this threat would require attacking mobile DF21D missile launchers on the Chinese mainland. This would be highly escalatory, leading possibly to nuclear response.

4. The strategic importance of the South China Sea and artificial islands constructed there is primarily to the ability of the US to cut off the flow of oil to PRC. The islands may enable PRC to gain dominance in the region and make US submarine operations much more difficult. US reaction to these assets is not driven by "international law" or fishing or oil rights, or even the desire to keep shipping lanes open. What is at stake is the US capability to cut off oil flow, a non-nuclear but highly threatening card it has (until now?) had at its disposal to play against China.

The map below shows the consequences of full deployments of SAM, ASCM, and ASBM weaponry on the artificial islands. Consequences extend to the Malacca Strait (through which 80% of China's oil passes) and US basing in Singapore. Both linked articles are worth reading.

CHINA’S ARTIFICIAL ISLANDS ARE BIGGER (AND A BIGGER DEAL) THAN YOU THINK

Beijing's Go Big or Go Home Moment in the South China Sea



HAS CHINA BEEN PRACTICING PREEMPTIVE MISSILE STRIKES AGAINST U.S. BASES? (Lots of satellite photos at this link, revealing extensive ballistic missile tests against realistic targets.)



Terminal targeting of a moving aircraft carrier by an ASBM like the DF21D

Simple estimates: 10 min flight time means ~10km uncertainty in final position of a carrier (assume speed of 20-30 mph) initially located by satellite. Missile course correction at distance ~10km from target allows ~10s (assuming Mach 5-10 velocity) of maneuver, and requires only a modest angular correction. At this distance a 100m sized target has angular size ~0.01 so should be readily detectable from an optical image. (Carriers are visible to the naked eye from space!) Final targeting at distance ~km can use a combination of optical / IR / radar that makes countermeasures difficult.

So hitting a moving aircraft carrier does not seem especially challenging with modern technology. The Chinese can easily test their terminal targeting technology by trying to hit, say, a very large moving truck at their ballistic missile impact range, shown above.

I do not see any effective countermeasures, and despite inflated claims concerning anti-missile defense capabilities, it is extremely difficult to stop an incoming ballistic missile with maneuver capability.


More analysis and links to strategic reports from RAND and elsewhere in this earlier post The Pivot and American Statecraft in Asia.

... These questions of military/technological capability stand prior to the prattle of diplomats, policy analysts, or political scientists. Perhaps just as crucial is whether top US and Chinese leadership share the same beliefs on these issues.

... It's hard to war game a US-China pacific conflict, even a conventional one. How long before the US surface fleet is destroyed by ASBM/ASCM? How long until forward bases are? How long until US has to strike at targets on the mainland? How long do satellites survive? How long before the conflict goes nuclear? I wonder whether anyone knows the answers to these questions with high confidence -- even very basic ones, like how well asymmetric threats like ASBM/ASCM will perform under realistic conditions. These systems have never been tested in battle.

The stakes are so high that China can just continue to establish "facts on the ground" (like building new island bases), with some confidence that the US will hesitate to escalate. If, for example, both sides secretly believe (at the highest levels; seems that Xi is behaving as if he might) that ASBM/ASCM are very effective, then sailing a carrier group through the South China Sea becomes an act of symbolism with meaning only to those that are not in the know.

The scale of ... triumph cannot be exaggerated. He ... had brought about a complete transformation of the European international order. He had told those who would listen what he intended to do, how he intended to do it, and he did it. He achieved this incredible feat without commanding an army, and without the ability to give an order to the humblest common soldier, without control of a large party, without public support, indeed, in the face of almost universal hostility, without a majority in parliament, without control of his cabinet, and without a loyal following in the bureaucracy. --- Brexit: victory over the Hollow Men

Dominic Cummings finally reveals some of the details behind the Brexit campaign's victory last summer:

On the referendum #20: the campaign, physics and data science

‘If you don’t get this elementary, but mildly unnatural, mathematics of elementary probability into your repertoire, then you go through a long life like a one-legged man in an ass-kicking contest. You’re giving a huge advantage to everybody else. One of the advantages of a fellow like Buffett … is that he automatically thinks in terms of decision trees and the elementary math of permutations and combinations… It’s not that hard to learn. What is hard is to get so you use it routinely almost everyday of your life. The Fermat/Pascal system is dramatically consonant with the way that the world works. And it’s fundamental truth. So you simply have to have the technique…

‘One of the things that influenced me greatly was studying physics… If I were running the world, people who are qualified to do physics would not be allowed to elect out of taking it. I think that even people who aren’t [expecting to] go near physics and engineering learn a thinking system in physics that is not learned so well anywhere else… The tradition of always looking for the answer in the most fundamental way available – that is a great tradition.’ --- Charlie Munger, Warren Buffet’s partner.

During the ten week official campaign the implied probability from Betfair odds of IN winning ranged between 60-83% (rarely below 66%) and the probability of OUT winning ranged between 17-40% (rarely above 33%). One of the reasons why so few in London saw the result coming was that the use by campaigns of data is hard to track even if you know what to look for and few in politics or the media know what to look for yet. Almost all of Vote Leave’s digital communication and data science was invisible even if you read every single news story or column ever produced in the campaign or any of the books so far published (written pre-Shipman’s book).

... We created new software. This was a gamble but the whole campaign was a huge gamble and we had to take many calculated risks. One of our central ideas was that the campaign had to do things in the field of data that have never been done before. This included a) integrating data from social media, online advertising, websites, apps, canvassing, direct mail, polls, online fundraising, activist feedback, and some new things we tried such as a new way to do polling (about which I will write another time) and b) having experts in physics and machine learning do proper data science in the way only they can – i.e. far beyond the normal skills applied in political campaigns. We were the first campaign in the UK to put almost all our money into digital communication then have it partly controlled by people whose normal work was subjects like quantum information (combined with political input from Paul Stephenson and Henry de Zoete, and digital specialists AIQ). We could only do this properly if we had proper canvassing software. We built it partly in-house and partly using an external engineer who we sat in our office for months.

Many bigshot traditional advertising characters told us we were making a huge error. They were wrong. It is one of the reasons we won. We outperformed the IN campaign on data despite them starting with vast mounts of data while we started with almost zero, they had support from political parties while we did not, they had early access to the electoral roll while we did not, and they had the Crosby/Messina data and models from the 2015 election while we had to build everything from scratch without even the money to buy standard commercial databases (we found ways to scrape equivalents off the web saving hundreds of thousands of pounds).

If you want to make big improvements in communication, my advice is – hire physicists, not communications people from normal companies, and never believe what advertising companies tell you about ‘data’ unless you can independently verify it. Physics, mathematics, and computer science are domains in which there are real experts, unlike macro-economic forecasting which satisfies neither of the necessary conditions – 1) enough structure in the information to enable good predictions, 2) conditions for good fast feedback and learning. Physicists and mathematicians regularly invade other fields but other fields do not invade theirs so we can see which fields are hardest for very talented people. It is no surprise that they can successfully invade politics and devise things that rout those who wrongly think they know what they are doing. Vote Leave paid very close attention to real experts. ...

More important than technology is the mindset – the hard discipline of obeying Richard Feynman’s advice: ‘The most important thing is not to fool yourself and you are the easiest person to fool.’ They were a hard floor on ‘fooling yourself’ and I empowered them to challenge everybody including me. They saved me from many bad decisions even though they had zero experience in politics and they forced me to change how I made important decisions like what got what money. We either operated scientifically or knew we were not, which is itself very useful knowledge. (One of the things they did was review the entire literature to see what reliable studies have been done on ‘what works’ in politics and what numbers are reliable.) Charlie Munger is one half of the most successful investment partnership in world history. He advises people – hire physicists. It works and the real prize is not the technology but a culture of making decisions in a rational way and systematically avoiding normal ways of fooling yourself as much as possible. This is very far from normal politics.

...

On the eve and day of Brexit I happened to be staying at the estate of a billionaire hedge fund manager, which hosted a meeting of elite capital allocators. At breakfast, more than half of these titans of capital were in shock (others, entirely unperturbed :-) ... Markets were down 8% or more and my host asked for my view. It will play out over years, I said. No one knows where this is going to go. The market is oversold and it's a buying opportunity. So it was.


The Unreasonable Effectiveness of Mathematics in the Natural Sciences, Eugene Wigner:

... it is useful, in epistemological discussions, to abandon the idealization that the level of human intelligence has a singular position on an absolute scale. In some cases it may even be useful to consider the attainment which is possible at the level of the intelligence of some other species.

The impacts of studies ranked in the 3rd to 20th percentile are more or less statistically indistinguishable. With current funding lines as low as 10th percentile, this means that many unfunded proposals are better than funded studies.

NIH peer review percentile scores are poorly predictive of grant productivity
DOI: 10.7554/eLife.13323.001

Peer review is widely used to assess grant applications so that the highest ranked applications can be funded. A number of studies have questioned the ability of peer review panels to predict the productivity of applications, but a recent analysis of grants funded by the National Institutes of Health (NIH) in the US found that the percentile scores awarded by peer review panels correlated with productivity as measured by citations of grant-supported publications. Here, based on a re-analysis of these data for the 102,740 funded grants with percentile scores of 20 or better, we report that these percentile scores are a poor discriminator of productivity. This underscores the limitations of peer review as a means of assessing grant applications in an era when typical success rates are often as low as about 10%.

Scott Alexander (Slate Star Codex) on Philip Tetlock's Superforecasting:

Book review
Highlighted passages.

I especially liked this passage that Scott highlights:

When hospitals created cardiac care units to treat patients recovering from heart attacks, Cochrane proposed a randomized trial to determine whether the new units delivered better results than the old treatment, which was to send the patient home for monitoring and bed rest. Physicians balked. It was obvious the cardiac care units were superior, they said, and denying patients the best care would be unethical. But Cochrane was not a man to back down…he got his trial: some patients, randomly selected, were sent to the cardiac care units while others were sent home for monitoring and bed rest. Partway through the trial, Cochrane met with a group of the cardiologists who had tried to stop his experiment. He told them that he had preliminary results. The difference in outcomes between the two treatments was not statistically signficant, he emphasized, but it appeared that patients might do slightly better in the cardiac care units. “They were vociferous in their abuse: ‘Archie,’ they said, ‘we always thought you were unethical. You must stop the trial at once.'” But then Cochrane revealed he had played a little trick. He had reversed the results: home care had done slightly better than the cardiac units. “There was dead silence and I felt rather sick because they were, after all, my medical colleagues.”

[ Cochrane Collaboration ] [ Bounded Cognition ]

See also Medical Science?

Jason Zweig writes about Philip Tetlock's Good Judgement Project below. See also Expert Predictions, Perils of Prediction, and this podcast talk by Tetlock.

A quick summary: good amateurs (i.e., smart people who think probabilistically and are well read) typically perform as well as or better than area experts (e.g., PhDs in Social Science, History, Government; MBAs) when it comes to predicting real world outcomes. The marginal returns (in predictive power) to special "expertise" in soft subjects are small. (Most of the returns are in the form of credentialing or signaling ;-)

WSJ: ... I think Philip Tetlock’s “Superforecasting: The Art and Science of Prediction,” co-written with the journalist Dan Gardner, is the most important book on decision making since Daniel Kahneman’s “Thinking, Fast and Slow.” (I helped write and edit the Kahneman book but receive no royalties from it.) Prof. Kahneman agrees. “It’s a manual to systematic thinking in the real world,” he told me. “This book shows that under the right conditions regular people are capable of improving their judgment enough to beat the professionals at their own game.”

The book is so powerful because Prof. Tetlock, a psychologist and professor of management at the University of Pennsylvania’s Wharton School, has a remarkable trove of data. He has just concluded the first stage of what he calls the Good Judgment Project, which pitted some 20,000 amateur forecasters against some of the most knowledgeable experts in the world.

The amateurs won — hands down. Their forecasts were more accurate more often, and the confidence they had in their forecasts — as measured by the odds they set on being right — was more accurately tuned.

The top 2%, whom Prof. Tetlock dubs “superforecasters,” have above-average — but rarely genius-level — intelligence. Many are mathematicians, scientists or software engineers; but among the others are a pharmacist, a Pilates instructor, a caseworker for the Pennsylvania state welfare department and a Canadian underwater-hockey coach.

The forecasters competed online against four other teams and against government intelligence experts to answer nearly 500 questions over the course of four years: Will the president of Tunisia go into exile in the next month? Will the gold price exceed 1,850ドル on Sept. 30, 2011? Will OPEC agree to cut its oil output at or before its November 2014 meeting?

It turned out that, after rigorous statistical controls, the elite amateurs were on average about 30% more accurate than the experts with access to classified information. What’s more, the full pool of amateurs also outperformed the experts. ...

In technical subjects, such as chemistry or physics or mathematics, experts vastly outperform lay people even on questions related to everyday natural phenomena (let alone specialized topics). See, e.g., examples in Thinking Physics or Physics for Future Presidents. Because these fields have access to deep and challenging questions with demonstrably correct answers, the ability to answer these questions (a combination of cognitive ability and knowledge) is an obviously real and useful construct. See earlier post The Differences are Enormous:

Luis Alvarez laid it out bluntly:

The world of mathematics and theoretical physics is hierarchical. That was my first exposure to it. There's a limit beyond which one cannot progress. The differences between the limiting abilities of those on successively higher steps of the pyramid are enormous.

... People who work in "soft" fields (even in science) don't seem to understand this stark reality. I believe it is because their fields do not have ready access to right and wrong answers to deep questions. When those are available, huge differences in cognitive power are undeniable, as is the utility of this power.

Thought experiment for physicists: imagine a professor throwing copies of Jackson's Classical Electrodynamics at a group of students with the order, "Work out the last problem in each chapter and hand in your solutions to me on Monday!" I suspect that this exercise produces a highly useful rank ordering within the group, with huge differences in number of correct solutions.

Highly recommended podcast: Tim Harford (FT) at the LSE. Among the topics covered are Keynes' and Irving Fisher's performance as investors, and Philip Tetlock's IARPA-sponsored Good Judgement Project, meant to evaluate expert prediction of complex events. Project researchers (psychologists) find that "actively open-minded thinkers" (those who are willing to learn from those that disagree with them) perform best. Unfortunately there are no real "super-predictors" -- just some who are better than others, and have better calibration (accurate confidence estimates).


[フレーム]

[フレーム]

Highly recommended talk by Dominic Cummings at the UK Institute for Public Policy Research. See also The Hollow Men (original essay).

Podcast version here and below.

[フレーム]

In the previous post Working in the dark, I questioned the validity (predictive power; terminology from psychometrics) of holistic admissions used by elite universities. Below is an example of validity: the football recruiting star system for HS seniors, measuring its ability to predict performance in college (chance of being named to a college All-American team). All players in the dataset below were signed by Division I football programs -- they are elite, recruited athletes.



You can see that there is significant predictive efficiency gained in going from 2 to 3 or 3 to 4 stars. For example, there are only about a third more 2 star recruits than 3 star recruits, but the latter are almost 10x as likely to be named All-American. On the other hand, the distinction between 4 and 5 star recruits seems a bit iffy to me. Only about 1 in 10 players at or above the 4 star level are designated 5 star, with the latter distinction raising their All-America odds by a bit less than 3. There's information there, but you pay a high price in selectivity.

It's not surprising that this system works. Coaches are heavily incentivized to win, and some ratings are provided by professionals who make their living scouting HS talent. But of course prediction is imperfect: there are plenty of 3 and 4 star recruits who outperform 5 stars. The issue is whether expected return from using the predictions is positive...

University admissions committees should be able to produce an analysis of this quality or better. If the objective function to be optimized is performance at the university, the data is readily available (see below). But one could also use criteria such as eventual net worth (major donor status), fame, notable achievements, etc. to assess quality of admissions decisions. Lots of assertions are made in this context (see comments on this NYTimes article), such as that high test scores are negatively correlated with leadership or interpersonal skills that impact later life. This might be true, but I've yet to find any careful analysis of the claim.

How much can we enhance odds ratios for becoming a millionaire / billionaire / STEM PhD / Nobel laureate by proper filtering of applicants?

See Data mining the university and Nonlinear psychometric thresholds for physics and mathematics for predictors of college performance by major as a function of HS GPA and SAT.

Holistic evaluation of applicants = noise? (Or worse?)

Why no evidence-based admissions?

Is there any serious study of whether subjective criteria used to judge applicants actually predict success? In psychometrics this is referred to as test validity. In the article below, it is not even clear that the evaluation method satisfies the weaker criteria of consistency or stability: applicants passed through the system another time might generate significantly different scores. "Expert" evaluation often reduces the power of prediction relative to simple algorithms.

See Data mining the university and Nonlinear psychometric thresholds for physics and mathematics for plenty of evidence of validity, consistency and stability of traditional measures of intellectual ability.

NYTimes: A highly qualified student, with a 3.95 unweighted grade point average and 2300 on the SAT, was not among the top-ranked engineering applicants to the University of California, Berkeley. He had perfect 800s on his subject tests in math and chemistry, a score of 5 on five Advanced Placement exams, musical talent and, in one of two personal statements, had written a loving tribute to his parents, who had emigrated from India.

Why was he not top-ranked by the “world’s premier public university,” as Berkeley calls itself? Perhaps others had perfect grades and scores? They did indeed. Were they ranked higher? Not necessarily. What kind of student was ranked higher? Every case is different.

The reason our budding engineer was a 2 on a 1-to-5 scale (1 being highest) has to do with Berkeley’s holistic, or comprehensive, review, an admissions policy adopted by most selective colleges and universities. In holistic review, institutions look beyond grades and scores to determine academic potential, drive and leadership abilities. Apparently, our Indian-American student needed more extracurricular activities and engineering awards to be ranked a 1.

Now consider a second engineering applicant, a Mexican-American student with a moving, well-written essay but a 3.4 G.P.A. and SATs below 1800. His school offered no A.P. He competed in track when not at his after-school job, working the fields with his parents. His score? 2.5.

Both students were among “typical” applicants used as norms to train application readers like myself. And their different credentials yet remarkably close rankings illustrate the challenges, the ambiguities and the agenda of admissions at a major public research university in a post-affirmative-action world.

[ Despite Prop. 209, the nearly equal scores of these "typical" training cases suggests outcomes very similar to those produced by explicit affirmative action. ]

... I could see the fundamental unevenness in this process both in the norming Webinars and when alone in a dark room at home with my Berkeley-issued netbook, reading assigned applications away from enormously curious family members. First and foremost, the process is confusingly subjective, despite all the objective criteria I was trained to examine.

In norming sessions, I remember how lead readers would raise a candidate’s ranking because he or she “helped build the class.”

... After the next training session, when I asked about an Asian student who I thought was a 2 but had only received a 3, the officer noted: “Oh, you’ll get a lot of them.” She said the same when I asked why a low-income student with top grades and scores, and who had served in the Israeli army, was a 3. ...

Pundit idiocracy: "Close race", "Too close to call", "Neck and neck". (I heard this all day long.)

Quants and data geeks: "Obama will win. Unlikely to be close."

From an earlier post High V, Low M:

high verbal ability ... is useful for appearing to be smart, or for winning arguments and impressing other people, but it's really high math ability that is useful for discovering things about the world -- that is, discovering truth or reasoning rigorously.

... The statistical techniques used to analyze data obtained in a messy, complex world require mathematical ability to practice correctly. In almost all realistic circumstances hypothesis testing is intrinsically mathematical.

See also Obama wins! and Expert Prediction. Scorecard of predictions here (accuracy highly correlated with M, not V ;-)

Who is this guy?

Xu Cheng, Moodys’ Analytics: Obama 303, Romney 235 (Note that this prediction was made back in February) “This prediction is tied to the Moody’s Analytics current baseline forecast for U.S. growth, which assumes that most states will continue to recover at slow to moderate speeds.”

At least, according to the quants who performed the mind boggling, incomprehensible, mysterious, nearly impossible task of averaging state poll numbers to estimate likely electoral vote totals.

Pundit and non-quant reactions evidence of Idiocracy. See earlier post Bounded Cognition.

Chronicle: ... While it may not seem likely, poll aggregation is a threat to the supremacy of the punditocracy. In the past week, you could sense that some high-profile media types were being made slightly uncomfortable by the bespectacled quants, with their confusing mathematical models and zippy computer programs. The New York Times columnist David Brooks said pollsters who offered projections were citizens of “sillyland.”

Maybe, but the recent track record in sillyland is awfully solid. In the 2008 presidential election, Silver correctly predicted 49 of 50 states. Wang was off by only one electoral vote. Meanwhile, as Silver writes in his book, numerous pundits confidently predicted a John McCain victory based on little more than intestinal twinges.

... Most journalists are ill equipped to interpret data, he says (and few journalists would disagree), so they view statistics with skepticism and occasionally, in the case of Brooks, disdain. “The data-driven people are going to win in the long run,” Jackman says.

He sees it as part of the rise of what’s being called Big Data—that is, using actual information to make decisions. As Jackman points out, Big Data is already changing sports and business, and it may be that pundits are the equivalents of the baseball scouts in Michael Lewis’s book Moneyball, caring more about the naturalness of a hitter’s swing than whether he gets on base.

“Why,” Jackman wonders, “should political commentary be exempt from this movement?”

... Last week the professional pundit and MSNBC host Joe Scarborough ranted that people like Silver, Wang, Linzer, and Jackman—who think the presidential race is “anything but a tossup”—should be kept away from their computers “because they’re jokes.” Silver responded by challenging Scarborough to bet 1,000ドル on Romney (in the form of a donation to the American Red Cross) if he was so sure. This led to hand-wringing about whether it was appropriate for someone affiliated with The New York Times to make crass public wagers.

But the bet seemed like an important symbolic moment. The poll aggregators have skin in the game. They’ve made statistical forecasts and published them, not just gut-feeling guesses on Sunday-morning talk shows. And, in Silver’s case, as a former professional poker player, he is willing to back it up with something tangible.

Alex Tabarrok, an economist and blogger for Marginal Revolution, applauded, calling such bets a “tax on bullshit.” ...

Shout out to Sam Wang, Caltech '86 :-)

I recently came across this excellent talk (podcast number 84 on the list at the link) by Philip Tetlock about his research on expert prediction.

Putting aside the fox vs hedgehog dichotomy, I think the main takeaway is that "expert" predictions are no better than those of well-informed ordinary people, and barely outperform simple algorithms.

Longnow.org: ... Tetlock took advantage of getting tenure to start a long-term research project now 18 years old to examine in detail the outcomes of expert political forecasts about international affairs. He studied the aggregate accuracy of 284 experts making 28,000 forecasts, looking for pattern in their comparative success rates. Most of the findings were negative— conservatives did no better or worse than liberals; optimists did no better or worse than pessimists. Only one pattern emerged consistently.

“How you think matters more than what you think.”

It’s a matter of judgement style, first expressed by the ancient Greek warrior poet Archilochus: “The fox knows many things; the hedgehog one great thing.” The idea was later expanded by essayist Isaiah Berlin. In Tetlock’s interpretation, Hedgehogs have one grand theory (Marxist, Libertarian, whatever) which they are happy to extend into many domains, relishing its parsimony, and expressing their views with great confidence. Foxes, on the other hand are skeptical about grand theories, diffident in their forecasts, and ready to adjust their ideas based on actual events.

The aggregate success rate of Foxes is significantly greater, Tetlock found, especially in short-term forecasts. And Hedgehogs routinely fare worse than Foxes, especially in long-term forecasts. They even fare worse than normal attention-paying dilletantes — apparently blinded by their extensive expertise and beautiful theory. Furthermore, Foxes win not only in the accuracy of their predictions but also the accuracy of the likelihood they assign to their predictions— in this they are closer to the admirable discipline of weather forecasters.

The value of Hedgehogs is that they occasionally get right the farthest-out predictions— civil war in Yugoslavia, Saddam’s invasion of Kuwait, the collapse of the Internet Bubble. But that comes at the cost of a great many wrong far-out predictions— Dow 36,000, global depression, nuclear attack by developing nations.

Hedgehogs annoy only their political opposition, while Foxes annoy across the political spectrum, in part because the smartest Foxes cherry-pick idea fragments from the whole array of Hedgehogs.

Bottom line… The political expert who bores you with an cloud of “howevers” is probably right about what’s going to happen. The charismatic expert who exudes confidence and has a great story to tell is probably wrong.

And to improve the quality of your own predictions, keep brutally honest score. Enjoy being wrong, admitting to it and learning from it, as much as you enjoy being right.

See also Intellectual honesty: how much do we know?

Daniel Kahneman claims that differences in the performance of professional investors are mostly due to luck, whereas compensation is awarded as if differences are due to skill. Most alpha is fake alpha.

This of course raises all sorts of questions about why such people are allowed to become so extravagantly wealthy. The usual argument is that their investment decisions lead to more efficient resource allocation in the economy. (They are a "necessary evil" of a capitalist market system that benefits all of us :-) But if the decisions of the highest paid professionals are no better than those of average professionals, we could replace the services of the highest earners at much lower cost (or cap their salaries or impose high marginal tax rates) without negatively impacting the overall quality of decisions or the efficiency of the economy.








The article is worth reading in its entirety.

NYTimes: ... No one in the firm seemed to be aware of the nature of the game that its stock pickers were playing. The advisers themselves felt they were competent professionals performing a task that was difficult but not impossible, and their superiors agreed. On the evening before the seminar, Richard Thaler and I had dinner with some of the top executives of the firm, the people who decide on the size of bonuses. We asked them to guess the year-to-year correlation in the rankings of individual advisers. They thought they knew what was coming and smiled as they said, “not very high” or “performance certainly fluctuates.” It quickly became clear, however, that no one expected the average correlation to be zero.

What we told the directors of the firm was that, at least when it came to building portfolios, the firm was rewarding luck as if it were skill. This should have been shocking news to them, but it was not. There was no sign that they disbelieved us. How could they? After all, we had analyzed their own results, and they were certainly sophisticated enough to appreciate their implications, which we politely refrained from spelling out. We all went on calmly with our dinner, and I am quite sure that both our findings and their implications were quickly swept under the rug and that life in the firm went on just as before. The illusion of skill is not only an individual aberration; it is deeply ingrained in the culture of the industry. Facts that challenge such basic assumptions — and thereby threaten people’s livelihood and self-esteem — are simply not absorbed. The mind does not digest them. This is particularly true of statistical studies of performance, which provide general facts that people will ignore if they conflict with their personal experience.

The next morning, we reported the findings to the advisers, and their response was equally bland. Their personal experience of exercising careful professional judgment on complex problems was far more compelling to them than an obscure statistical result. When we were done, one executive I dined with the previous evening drove me to the airport. He told me, with a trace of defensiveness, “I have done very well for the firm, and no one can take that away from me.” I smiled and said nothing. But I thought, privately: Well, I took it away from you this morning. If your success was due mostly to chance, how much credit are you entitled to take for it?

From the comments.

If you read the whole article, you see that Kahneman does believe in skill. For example, his studies show that some doctors are better at diagnosis than others. I am also sure that some entrepreneurs or some physicists or some athletes are better than others. (Although in the case of entrepreneurs it would be very hard to demonstrate statistically since outcomes are noisy and the number of attempts per entrepreneur is relatively small.)

But there may be areas where *the differences between high level professionals* (e.g., people who have been hired to run money, have top MBAs or graduate degrees, etc.) are statistically seen to be mostly due to luck. This has already been convincingly demonstrated for pundits or analysts of complex world events by Tetlock's studies of expertise. (You can find several posts on this blog on the topic.) Whether it's true of money managers (or even big company CEOs) is controversial. If you argue the skill side, I'd like to see *your* statistical evidence, not just repetition of your priors (again and again).

Expert predictions

In all areas of human activity, even the skill dominated ones, luck plays a big factor. This is a good argument for redistribution -- almost every successful person owes some of their success to luck.

and

It seems to me that the 20th century trend in democracies is toward greater redistribution: social safety nets, guaranteed minimum income, etc. People have been conditioned to believe these are aspects of a just society.

The question is: what is the optimum level of redistribution? (Given a particular utility function for society.)

One argument is that we have to let the rich get rich in order to have strong economic growth. Too much redistribution means a smaller pie to split. But the Illusion of Skill argument (if correct) suggests that for some activities like finance a high marginal tax rate (say, which kicks in above the income of the *average* finance professional; this would then only affect the top earning financiers who, according to the argument are not adding any real value that the average guys can't also provide) would not negatively affect economic efficiency.

If people irrationally and incorrectly believe that only Harvard MDs are capable of treating pneumonia, and bid up their compensation to exorbitant levels (levels so high that the Harvard MDs begin exerting financial and political control over society as a whole), wouldn't it be better for society to impose a high tax rate on Harvard MDs, which kicks in above the income of other doctors with similar credentials (but who are not beneficiaries of the irrational belief)?

I came across this compendium of Mark Zandi (Chief Economist, Moody's Analytics) predictions. I've read and heard many interviews with Zandi over the years, and he always seems like a thoughtful guy. But it's not surprising that he, like other experts, has no more predictive power concerning complex events than monkeys throwing darts.

What is amazing to me is how Zandi can remain so confident about his capabilities when sufficient evidence has accumulated to the contrary. I suppose talking heads, pundits, market economists (and even some successful scientists and corporate leaders) are selected specifically for this kind of overconfidence. There's no room for intellectual honesty when you're trying to get ahead ;-)

Ritholtz: The Fed, Treasury and the Senate Budget Committee appear to have a favorite private sector economist, one who has managed to become a favorite even though he works for a unit of the same rating agency whose analysis is intrinsically tied to both the market, banking and housing crisis.

Mark Zandi of Moody’s Economy.com is routinely trotted out as an independent expert. He was the sole economist at the August 17 Treasury Conference on the Future of Housing Finance, the Fed’s REO and Vacant Properties conference and has now testified at the September 22nd Senate Budget Committee hearing on “Assessing the Federal Policy Response to the Economic Crisis”.

Never mind that, based on Zandi’s record, either his analysis is just wrong or his independence is compromised. Everyone seems to like to hear the guy who is saying what people want to hear, even the press appears to prefer “feel good” analysis to considering the accuracy of his record. ...


“It’s at least three or four quarters before we see the bottom of the housing market,” Zandi said.

Wire & Staff Reports – Oct 27, 2006

“The housing market correction is in full swing but it probably has another year to go before it bottoms out,” said Mark Zandi, chief economist at Moody’s Economy.com.

Los Angeles Times – Jan 6, 2007

“Mark Zandi, chief economist for Moody’s Economy.com, said jobs and wages were growing too fast for their own good. He warned that higher wages could induce companies to raise prices, which could lead workers to demand higher wages — an inflationary wage-price spiral.”

[Note: The chart below shows 3-mo. Changes in total civilian compensation. It seems not to demonstrate any wage-price spiral:

Marketwatch - March 26, 2007

"Zandi sees a bottom for sales in spring as sellers become more motivated and start cutting prices." [Note: In August 2010, new home sales fell to the lowest level since 1963, when the government began to keep records.]

New York Times – March 18, 2007

“Weakness in the market has been concentrated in certain segments,” says Mark Zandi, chief economist of Moody’s Economy.com. “We’re not witnessing the entire housing market in metro areas caving in.”

MTG Foundation – April 28, 2007

...