Notebooks

Artificial Intelligence

Last update: 21 Aug 2025 14:08
First version: Sometime before 13 March 1995

Yet Another Inadequate Placeholder.

I am not best-pleased to see this phrase come back in to vogue over the last few years, riding on a combination of absurd, apocalyptic myth-making and real but limited advances in the art of curve-fitting, a.k.a. "deep learning". (Said differently, I remember the last time Geoff Hinton's students were going to take over the world with multi-layer connectionist models.)

    Recommended, close-ups:
  • Margaret Boden, The Creative Mind: Myths and Mechanisms
  • Maciej Ceglowski, "Superintelligence: The Idea That Eats Smart People" (29 October 2016)
  • Daniel Dennett
  • Marco Dorigo and Marco Colombetti, Robot Shaping: An Experiment in Behavior Engineering
  • The Genealogy of ELIZA [I am not altogether joking when I say that you should not trust commentary on AI from anyone who has not both interacted with Eliza, and stepped through the source code.]
  • John Holland, Adaptation in Natural and Artificial Systems
  • Gary Marcus
  • Drew McDermott, "Artificial Intelligence Meets Natural Stupidity", ACM SIGART Bulletin 57 (April, 1976): 4--9
  • Melanie Mitchell
    • "Why AI is Harder Than We Think", arxiv:2104.12871
    • "Do half of AI researchers believe that there's a 10% chance AI will kill us all?", 23 April 2023 [Shorter and less polite MM: No, because that's preposterous.]
    • "How do we know how smart AI systems are?", Science 381 (2023): adj5957
  • Adam Sobieszek and Tadeusz Price, "Playing Games with AIs: The Limits of GPT-3 and Similar Large Language Models", Minds and Machines 32 341--364 [I have a bunch of quibbles and comments. (0) Gosh, there are a lot of citations to the journal editors. (1) They don't actually use item response theory! They just suggest that we can get information about whether a question-answerer is a human or a computer if different sources have different probabilities of an answer. Which is absolutely true and maybe worth mentioning in this context, but doesn't need IRT. (I say this as someone who thinks that a Rasch model for Turing tests would be awesome.) There is also the issue of why we should think there would be a probability of a given answer for either human beings or machines, stable over time. (2) I think their information-theory-inspired remarks about compression and generalization are a bit over-simplified. But I realize I have (or once had) expert over-sensitivity in this area, and I guess what they say is close enough to right for present purposes. (3) I think the point that if you are going to learn to predict unlabeled text, you are not going to be able to distinguish truth from falsehood, is quite right. (Even true texts are going to contain refutations, hypotheticals, etc.). (4) Similarly, the idea that statistical properties of symbol strings complicate the syntax/semantics distinction is one I remember being fairly widely understood in the late 1990s. (I'd argue that you can find a version of it in Zellig Harris's Language and Information .) Certainly in my own (then) area of research, if in a particular stochastic process the string "01" is followed by "1" 99% of the time, it's very hard to avoid saying things like "'01' usually implies '1' is coming" (cf. "black clouds approaching mean rain soon"). But the semantic field (if I may put it that way) is limited to more of the same process, not the rest of the world. (5) It is incredibly striking to me that there is absolutely nothing in this paper about the specific architectures of the neural networks involved, other than their ability to maintain some sort of long-range context. If these arguments were right, we should be able to do the same thing with a sufficiently powered-up implementation of any probabilistic text predictor --- maybe even anything which does universal source coding. If someone is looking for a nice (but expensive) project, then, implementing one of Paul Algoet's old universal prediction schemes at read-the-whole-Web scale suggests itself.]
  • Judea Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference [Graphical models, before Uncle Judea got seized by the importance of causality]
  • Arjun Ramani and Zhengdong Wang, "Why transformative artificial intelligence is really, really hard to achieve", The Gradient 26 June 2023
  • Roger Schank, Tell Me a Story: A New Look at Real and Artificial Memory [Comments]
  • Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction
    Recommended, historical:
  • Allen Newell and Herbert A. Simon, "Current Developments in Complex Information Processing", RAND Corporation report P-850, 1 May 1956 [Thanks to Chris Wiggins for sharing a copy with me --- it's probably available online somewhere... --- and indeed "somewhere" proves to be CMU!]
  • J. McCarthy, M. L. Minsky, N. Rochester and C. E. Shannon, "Proposal for the Dartmouth Summer Research Project on Artificial Intelligence" [1955; PDF scan via Ray Solomonoff (!)]
  • Claude E. Shannon and John McCarthy (eds.), Automata Studies (1956)
  • Herbert A. Simon
    • Models of My Life
    • The Shape of Automation for Men and Management = The New Science of Management Decisions
    To read, popularizations [since this is related to my teaching]
  • Meredith Broussard, Artificial Unintelligence: How Computers Misunderstand the World
  • Melanie Mitchell, Artificial Intelligence: A Guide for Thinking Humans
  • Janelle Shane, "You Look Like a Thing and I Love You": How AI Works and Why It's Making the World a Weirder Place

permanent link for this note RSS feed for this note

Notebooks : Powered by Blosxom

AltStyle によって変換されたページ (->オリジナル) /