Structural insight

Thus quantum mechanics occupies a very unusual place among physical theories: it contains classical mechanics as a limiting case, yet at the same time it requires this limiting case for its own formulation.

— Lev Landau and Evgeny Lifshitz, Quantum Mechanics: Non-relativistic Theory (3rd edition, 1977, afaik).

Gradually, across a series of posts exploring alternative structures for a basic theory of physics, I've been trying to tease together a strategy wherein quantum mechanics is, rather than a nondeterministic foundation of reality, an approximation valid for sufficiently small systems. This post considers how one might devise a concrete mathematical demonstration that the strategy can actually work.

I came into all this with a gnawing sense that modern physics had taken a conceptual wrong turn somewhere, that it had made some —unidentified— incautious structural assumption that ought not have been made and was leading it further and further astray. (I explored the philosophy of this at some depth in an earlier post in the series, several years ago by now.) The larger agenda here is to shake up our thinking on basic physics, accumulating different ways to structure theories so that our structural choices are made with eyes open, rather than just because we can't imagine an alternative. The particular notion I'm stalking atm —woven around the concept of co-hygiene, to be explained below— is, in its essence, that quantum mechanics might be an approximation, just as Newtonian mechanics is, and that the quantum approximation may be a consequence of the systems-of-interest being almost infinitesimally small compared to the cosmos as a whole. Quantum mechanics suggests that all the elementary parts of the cosmos are connected to all the other elementary parts, which is clearly not conducive to practical calculations. In the model I'm pursuing, each element is connected to just a comparatively few others, and the whole jostles about, with each adjustment to an element shuffling its remote connections so that over many adjustments the element gets exposed to many other elements. Conjecturally, if a sufficiently small system interacts in this way with a sufficiently vast cosmos, the resulting behavior of the small system could look a lot like nondeterminism.

The question is, could it look like quantum mechanics?

As I've remarked before, my usual approach to these sorts of posts is to lift down off my metaphorical shelf the assorted fragments I've got on the topic of interest; lay out the pieces on the table, adding at the same time any new bits I've lately collected; inspect them all severally and collectively, rearranging them and looking for new patterns as I see them all afresh; and record my trail of thought as I do so. Sometimes I find that since the last time I visited things, my whole perception of them has shifted (I was, for example, struck in a recent post by how profoundly my perception of Church's λ-calculus has changed just in the past several years). Hopefully I glean a few new insights from the fresh inspection, some of which find their way into the new groupings destined to go back up on the shelf to await the next time, while some other, more speculative branches of reasoning that don't make it into my main stream of thought are preserved in my record for possible later pursuit.

Moreover, each iteration achieves focus by developing some particular theme within its line of speculation; some details of previous iterations are winnowed away to allow an uncluttered view of the current theme; and once the new iteration reaches its more-or-less-coherent insights, such as they are, a reset is then wanted, to unclutter the next iteration. Most of the posts in this series —with a couple of exceptions (1, 2)— have focused on the broad structure of the cosmos, touching only lightly on concrete mathematics of modern physics that, after all, I've suspected from the start of favoring incautious structural assumptions. This incremental shifting between posts is why, within my larger series on physics, the current post has a transitional focus: reviewing the chosen cosmological structure in order to apply it to the abstract structure of the mathematics, preparing from abstract ground to launch an assault on the concrete.

Though I'll reach a few conclusions here —oriented especially toward guidance for the next installment in the series— much of this is going to dwell on reasons why the problem is difficult, which if one isn't careful could create a certain pessimism toward the whole prospect. I'm moderately optimistic that the problem can be pried open, over a sufficient number of patient iterations of study. The formidable appearance of a mountain in-the-large oughtn't prevent us from looking for a way to climb it.

Contents
Co-hygiene
Primitive wave functions
Probability distributions
Quantum/classical interface
Genericity
The universe says 'hi'
The upper box

Co-hygiene

The schematic mathematical model I'm considering takes the cosmos to be a vast system of parts with two kinds of connections between them: local (geometry), and non-local (network). The system evolves by discrete transformational steps, which I conjecture may be selected based entirely on local criteria but, once selected, may draw information from both local and non-local connections and may have both local and non-local effects. The local part of all this would likely resemble classical physics.

When a transformation step is applied, its local effect must be handled in a way that doesn't corrupt the non-local network; that's called hygiene. If the non-local effect of a step doesn't perturb pre-existing local geometry, I call that co-hygiene. Transformation steps are not required in general to be co-hygienic; but if they are, then local geometry is only affected by local transformation steps, giving the steps a close apparent affinity with the local geometry, and I conjectured this could explain why gravity seems more integrated with spacetime than do the other fundamental forces. (Indeed, wondering why gravity would differ from the other fundamental forces was what led me into the whole avenue of exploration in the first place.)

Along the way, though, I also wondered if the non-local network could explain why the system deviated from "classical" behavior. Here I hit on an idea that offered a specific reason why quantum mechanics might be an approximation that works for very small systems. My inspiration for this sort of mathematical model was a class of variant λ-calculi (in fact, λ-calculus is co-hygienic, while in my dissertation I studied variant calculi that introduce non-co-hygienic operations to handle side-effects); and in those variant calculi, the non-local network topology is highly volatile. That is, each time a small subsystem interacts non-locally with the rest of the system, it may end up with different network neighbors than it had before. This means that if you're looking at a subsystem that is smaller than the whole system by a cosmically vast amount — say, if the system as a whole is larger than the subsystem by a factor of 10⁷⁰ or 10⁸⁰ — you might perform a very large number of non-local interactions and never interact with the same network-neighbor twice. It would be, approximately, as if there were an endless supply of other parts of the system for you to interact non-locally with. Making the non-local interactions look rather random.

Without the network-scrambling, non-locality alone would not cause this sort of seeming-randomness. The subsystem of interest could "learn" about its network neighbors through repeated interaction with them, and they would become effectively just part of its internal state. Thus, the network-scrambling, together with the assumption that the system is vastly larger than the subsystem, would seem to allow the introduction of an element of effective nondeterminism into the model.

But, is it actually useful to introduce an element of effective nondeterminism into the model? Notwithstanding Einstein's remark about whether or not God plays dice, if you start with a classical system and naively introduce a random classical element into it, you don't end up with a quantum wave function. (There is a vein of research, broadly called stochastic electrodynamics, that seeks to derive quantum effects from classical electrodynamics with random zero-point radiation on the order of Planck's constant, but apparently they're having trouble accounting for some quantum effects, such as quantum interference.) To turn this seeming-nondeterminism to the purpose would require some more nuanced tactic.

There is, btw, an interesting element of flexibility in the sort of effective-nondeterminism introduced: The sort of mathematical model I'm conjecturing has deterministic rules, so conceivably there could be some sort of invariant properties across successive rearrangements of the network topology. Thus, some kinds of non-local influences could be seemingly-random while others might, at least under some particular kinds of transformation (such as, under a particular fundamental force), be constant. The subsystem of interest could "learn" these invariants through repeated interactions, even though other factors would remain unlearnable. In effect, these invariants would be part of the state of the subsystem, information that one would include in a description of the subsystem but that, in the underlying mathematical model, would be distributed across the network.

Primitive wave functions

Suppose we're considering some very small physical system, say a single electron in a potential field.

A potential field, as I suggested in a previous post, is a simple summation of combined influences of the rest of the cosmos on the system of interest, in this case our single electron. Classically —and under Relativity— the potential field would tell us nothing about non-local influences on the electron. In this sort of simple quantum-mechanical exercise, the potential field used is, apparently, classical.

The mathematical model in conventional quantum mechanics posits, as its underlying reality, a wave function — a complex- (or quaternion-, or whatever-) valued field over the state space of the system, obeying some wave equation such as Schrödinger's,

iℏ ∂ Ψ

∂t

= Ĥ Ψ .

This posited underlying reality has no electron in the classical sense of something that has a precise position and momentum at each given time; the wave function is what's "really" there, and any observation we would understand as measuring the position or momentum of the electron is actually drawing on the information contained in the wave function.

While the wave function evolves deterministically, the mathematical model as a whole presents a nondeterministic theory. This nondeterminism is not a necessary feature of the theory. An alternative mathematical model exists, giving exactly the same predictions, in which there is an electron there in the classical sense, with precise position and momentum at each given time. Of course its position and momentum can't be simultaneously known by an observer (which would violate the Heisenberg uncertainty principle); but in the underlying model the electron does have those unobservable attributes. David Bohm published this model in 1952. However Bohm's model doesn't seem to have offered anything except a demonstration that quantum theory does not prohibit the existence of an unobservable deterministic classical electron. In Bohm's model, the electron had a definite position and momentum, yes, but it was acted on by a "pilot wave" that, in essence, obeyed Schrödinger's equation. And Schrödinger's equation is non-local, in the sense that not only does it allow information (unobservable information) to propagate faster than light, it allows it to "propagate" infinitely fast; the hidden information in the wave function does not really propagate "through" space, it just shows up wherever the equation says it should. Some years later, Bell's Theorem would show that this sort of non-locality is a necessary feature of any theory that always gives the same predictions as quantum mechanics (given some other assumptions, one of which I'm violating; I'll get back to that below); but my main point atm is that Bohm's model doesn't offer any new way of looking at the wave function itself. You still have to just accept the wave function as a primitive; Bohm merely adds an extra stage of reasoning in understanding how the wave function applies to real situations. If there's any practical, as opposed to philosophical, advantage to using Bohm's model, it must be a subtle one. Nevertheless, it does reassure us that there is no prohibition against a model in which the electron is a definite, deterministic thing in the classical sense.

The sort of model I'm looking for would have two important differences from Bohm's.

First, the wave function would not be primitive at all, but instead would be a consequence of the way the local-geometric aspect of the cosmos is distorted by the new machinery I'm introducing. The Schrödinger equation, above, seems to have just this sort of structure, with Ĥ embodying the classical behavior of the system while the rest of the equation is the shape of the distorting lens through which the classical behavior passes to produce its quantum behavior. The trick is to imagine any sensible way of understanding this distorting lens as a consequence of some deeper representation (keeping in mind that the local-geometric aspect of the cosmos needn't be classical physics as such, though this would be one's first guess).

A model with different primitives is very likely to lead to different questions; to conjure a quote from Richard Feynman, "by putting the theory in a certain kind of framework you get an idea of what to change". Hence a theory in which the wave function is not primitive could offer valuable fresh perspective even if it isn't in itself experimentally distinguishable from quantum mechanics. There's also the matter of equivalent mathematical models that are easier or harder to apply to particular problems — conventional quantum mechanics is frankly hard to apply to almost any problem, so it's not hard to imagine an equivalent theory with different primitives could make some problems more tractable.

Second, the model I'm looking for wouldn't, at least not necessarily, always produce the same predictions as quantum mechanics. I'm supposing it would produce the same predictions for systems practically infinitesimal compared to the size of the cosmos. Whether or not the model would make experimentally distinguishable predictions from quantum mechanics at a cosmological scale, would seem to depend on how much, or little, we could work out about the non-local-network part of the model; perhaps we'd end up with an incomplete model where the network part of it is just unknown, and we'd be none the wiser (but for increased skepticism about some quantum predictions), or perhaps we'd find enough structural clues to conjecture a more specific model. Just possibly, we'd end up with some cosmological questions to distinguish possible network structures, which (as usual with questions) could be highly fruitful regardless of whether the speculations that led to the questions were to go down in flames, or, less spectacularly, were to produce all the same predictions as quantum mechanics after all.

Probability distributions

Wave functions have always made me think of probability distributions, as if there ought to be some deterministic thing underneath whose distribution of possible states is generating the wave function. What's missing is any explanation of how to generate a wave-function-like thing from a classical probability distribution. (Not to lose track of the terminology, this is classical in the sense of classical probability, which in turn is based on classical logic, rather than classical physics as such. Though they all come down to us from the late nineteenth century, and complement each other.)

A classical probability distribution, as such, is fairly distinctive. You have an observable with a range of possible values, and you have a range of possible worlds each of which induces an observable value. Each possible world has a non-negative-real likelihood. The (unnormalized) probability distribution for the observable is a curve over the range of observable values, summing for each observable value the likelihoods of all possible worlds that yield that observable value. The probability of the observable falling in a certain interval is the area under the curve over that interval, divided by the area under the curve over the entire range of observable values. If you add together two mutually disjoint sets of possibilities, the areas under their curves simply add, since for each observable value the set of possible worlds yielding it is just the ones in the first set and the ones in the second set.

The trouble is, that distinctive pattern of a classical probability distribution is not how wave functions work. When you add together two wave functions, the two curves get added all right, but the values aren't unsigned reals; they can cancel each other, producing an interference pattern as in classic electron diffraction. (I demonstrated the essential role of cancellation, and a very few other structural elements, in quantum mechanical behavior in a recent post.) As an additional plot twist, the wave function values add, but the probability isn't their sum but (traditionally) the square of the magnitude of their sum.

One solution is to reject classical logic, since classical logic gives rise to the addition rule for deterministic probability distributions. Just say the classical notion of logical disjunction (and conjunction, etc.) is wrong, and quantum logic is the way reality works. While you're at it, invoke the idea that the world doesn't have to make sense to us (I've remarked before on my dim view of the things beyond mortal comprehension trope). Whatever its philosophical merits or demerits, this approach doesn't fit the current context for two reasons: it treats the wave function as primitive whereas we're interested in alternative primitives, so it doesn't appear to get us anywhere new/useful; and, even if it did get us somewhere useful (which it apparently doesn't), it's not the class of mathematical model I'm exploring here. I'm pursuing a mathematical model spiritually descended from λ-calculus, which is very much in the classical deterministic tradition.

So, we're looking for a way to derive a wave function from a classical probability distribution. One has to be very canny about approaching something like this. It's not plausible this would be untrodden territory; the strategy would naturally suggest itself, and lots of very smart, highly trained physicists with strong motive to consider it have had nearly a century in which to do so. Yet, frankly, if anyone had succeeded it ought to be well-known in alternative-QM circles, and I'd hope to have at least heard of it. So going into the thing one should apply a sort of lamppost principle, and ask what one is bringing to the table that could possibly allow one to succeed where they did not. (A typical version of the lamppost principle would say, if you've lost your keys at night somewhere on a dark street with a single lamppost, you should look for them near the lamppost since your chances of finding them if they're somewhere else are negligible. Here, to mix the metaphors, the something-new you bring to the table is the location of your lamppost.)

I'm still boggled by how close the frontier of human knowledge is. In high school I chose computer science for a college major partly (though only partly) because it seemed to me like there was so much mathematics you could spend a lifetime on it without reaching the frontier — and yet, by my sophomore year in college I was exploring extracurricularly some odd corner of mathematics (I forget what, now) that had clearly never been explored before. And now I'm recently disembarked from a partly-mathematical dissertation; a doctoral dissertation being, rather by definition, stuff nobody has ever done before. The idea that the math I was doing in my dissertation was something nobody had ever done before, is just freaky. At any rate, I'm bringing to this puzzle in physics a mathematical perspective that's not only unusual for physics, but unique even in the branch of mathematics I brought it from.

The particular mathematical tools I'm mainly trying to apply are:

• "metatime" (or whatever else one wants to call it), over which the cosmos evolves by discrete transformation steps. This is the thing I'm doing that breaks the conditions for Bell's Theorem; but all I've shown it works for is reshaping a uniform probability distribution into one that violates Bell's Inequality (here), whereas now we're not just reshaping a particular distribution but trying to mess with the rules by which distributions combine.

  My earlier post on metatime was explicitly concerned with the fact that quantum-mechanical predictions, while non-local with respect to time, could still be local with respect to some orthogonal dimension ("metatime"). Atm I'm not centrally interested in strict locality with respect to metatime; but metatime still interests me as a potentially useful tactic for a mathematical model, offering a smooth way to convert a classical probability distribution into time-non-locality.

• transformation steps that aggressively scramble non-local network topology. This seems capable of supplying classical nondeterminism (apparently, on a small scale); but the apparent nondeterminism we're after isn't classical.

• a broad notion that the math will stop looking like a wave function whenever the network scrambling ceases to sufficiently approximate classical nondeterminism (which ought to happen at large scales). But this only suggests that the nondeterminism would be a necessary ingredient in extracting a wave function, without giving any hint of what would replace the wave function when the approximation fails.

These are some prominent new things I'm bringing to the table. At least the second and third are new. Metatime is a hot topic atm, under a different name (pseudo-time, I think), as a device of the transactional interpretation of QM (TI). Advocates recommend TI as eliminating the conceptual anomalies and problems of other interpretations — EPR paradox, Schrödinger's cat, etc. — which bodes well for the utility of metatime here. I don't figure TI bears directly on the current purpose though because, as best I can tell, TI retains the primitive wave function. (TI does make another cameo appearance, below.)

On the problem of deriving the wave function, I don't know of any previous work to draw on. There certainly could be something out there I've simply not happened to cross paths with, but I'm not sanguine of finding such; for the most part, the subject suffers from a common problem of extra-paradigm scientific explorations: researchers comparing the current paradigm to its predecessor are very likely to come to the subject with intense bias. Researchers within the paradigm take pains to show that the old paradigm is wrong; researchers outside the paradigm are few and idiosyncratic, likely to be stuck on either the old paradigm or some other peculiar idea.

The bias by researchers within the paradigm, btw, is an important survival adaptation of the scientific species. The great effectiveness of paradigm science — which benefits its evolutionary success — is in enabling researchers to focus sharply on problems within the paradigm by eliminating distracting questions about the merits of the paradigm; and therefore those distracting questions have to be crushed decisively whenever they arise. It's hard to say whether this bias is stronger in the first generation of scientists under a paradigm, who have to get it moving against resistance from its predecessor, or amongst their successors trained within the zealous framework inherited from the first generation; either way, the bias tends to produce a dearth of past research that would aid my current purpose.

A particularly active, and biased, area of extra-paradigm science is no-go theorems, theorems proving that certain alternatives to the prevailing paradigm cannot be made to work (cf. old post yonder). Researchers within the paradigm want no-go theorems to crush extra-paradigm alternatives once and for all, and proponents of that sort of crushing agenda are likely, in their enthusiasm, to overlook cases not covered by the formal no-go-result. Extra-paradigm researchers, in contrast, are likely to ferret out cases not covered by the result and concentrate on those cases, treating the no-go theorems as helpful hints on how to build alternative ideas rather than discouragement from doing so. The paradigm researchers are likely to respond poorly to this, and accuse the alternative-seekers of being more concerned with rejecting the paradigm than with any particular alternative. The whole exchange is likely to generate much more heat than light.

Quantum/classical interface

A classical probability distribution is made up of possibilities. One of them is, and the others are not; we merely don't know which one is. This is important because it means there's no way these possibilities could ever interact with each other; the one that is has nothing to interact with because in fact there are no other possibilities. That is, the other possibilities aren't; they exist only in our minds. This non-interaction is what makes the probability distribution classical. Therefore, in considering ways to derive our wave function from classical probability distributions, any two things in the wave function that interact with each other do not correspond to different classical possibilities.

It follows that quantum states — those things that can be superposed, interfere with each other, and partly cancel each other out — are not separated by a boundary between different classical possibilities. This does not, on the face of it, prohibit superposable elements from being prior or orthogonal to such boundaries, so that the mathematical model superposes entities of some sort and then applies them to a classical probability distribution (or applies the distribution to them). Also keep in mind, though we're striving for a model in which the wave function isn't primitive, we haven't pinned down yet what is primitive.

Now, the wave function isn't a thing. It isn't observable, and we introduce it into the mathematics only because it's useful. So if it also isn't primitive, one has to wonder whether it's even needed in the mathematics, or whether perhaps we're simply to replace it by something else. To get a handle on this, we need to look at how the wave function is actually used in applying quantum mechanics to physical systems; after all, one can't very well fashion a replacement for one part of a machine unless one understands how that part interacts with the rest of the machine.

The entire subject of quantum mechanics appears imho to be filled with over-interpretation; to the extent any progress has been made in understanding quantum mechanics over the past nearly-a-century, it's consisted largely in learning to prune unnecessary metaphysical underbrush so one has a somewhat better view of the theory.

The earliest, conventional "interpretation" of QM, the "Copenhagen interpretation", says properties of the physical system don't exist until observed. This, to be brutally honest, looks to me like a metaphysical statement without practical meaning. There is a related, but more practical, concept called contextuality; and an associated — though unfortunately technically messy — no-go theorem called the Kochen–Specker theorem, a.k.a. the Bell–Kochen–Specker theorem. This all relates to the Heisenberg uncertainty principle, which says that you can't know the exact position and momentum of a particle at the same time; the more you know about its position, the less you can know about its momentum, and vice versa. One might think this would be because the only way to measure the particle's position or momentum is to interact with it, which alters the particle because, well, because to every action there is an equal and opposite reaction. However, in the practical application of the wave function to a quantum-mechanical system, there doesn't appear to be any experimental apparatus within the quantum system for the equal-and-opposite-reaction to apply to. Instead, there's simply a wave function and then it collapses. Depending on what you choose to observe (say, the position or the momentum), it collapses differently, so that the unobservable internal state of the system actually remembers which you chose to observe. This property, that the (unobservable) internal state of the system changes as a result of what you choose to measure about it, is contextuality; and the Kochen–Specker theorem says a classical hidden-variable theory, consistent with QM, must be contextual (much as Bell's Theorem says it must be non-local). Remember Bohm's hidden-variable theory, in which the particle does have an unobservable exact position and momentum? Yeah. Besides being rampantly non-local, Bohm's model is also contextual: the particle's (unobservable, exact) position and momentum are guided by the wave function, and the wave-function is perturbed by the choice of measurement, therefore the particle's (unobservable, exact) position and momentum are also perturbed by the choice of measurement.

Bell, being of a later generation than Bohr and Einstein (and thus, perhaps, less invested in pre-quantum metaphysical ideas), managed not to be distracted by questions of what is or isn't "really there". His take on the situation was that the difficulty was in how to handle the interface between quantum reality and classical reality — not philosophically, but practically. To see this, consider the basic elements of an exercise in traditional QM (non-relativistic, driven by Schrödinger's equation):

• A set of parameters define the classical state of the system; these become inputs to the wave equation. [typo fixed]

• A Hamiltonian operator Ĥ embodies the classical dynamics of the system.

• Schrödinger's equation provides quantum distortion of the classical system.

• A Hermitian operator called an "observable" embodies the experimental apparatus used to observe the system.  The wave function collapses to an eigenstate of the observable.

The observable is the interface between the quantum system and the classical world of the physicist; and Bell ascribes the difficulty to this interface. Consider a standard double-slit experiment in which an electron gun fires electrons one at a time through the double slit at a CRT screen where each electron causes a scintillation. As long as you don't observe which slit the electron passes through, you get an interference pattern from the wave function passing through the two slits, and that is quantum behavior; but there's nothing in the wave function to suggest the discreteness of the resulting scintillation. That discreteness results from the wave function collapse due to the observable, the interface with classical physics — and that discreteness is an essential part of the described physical reality. Scan that again: in order to fully account for physical reality, the quantum system has to encompass only a part of reality, because the discrete aspect of reality is only provided by the interface between the quantum system and surrounding classical physics. It seems that we couldn't describe the entire universe using QM even if we wanted to because, without a classical observable to collapse the wave function, the discrete aspect of physical reality would be missing. (Notice, this account of the difficulty is essentially structural, with only the arbitrary use of the term observable for the Hermitian operator as a vestige of the history of philosophical angst over the "role of the observer". It's not that there isn't a problem, but that presenting the problem as if it were philosophical only gets in the way of resolving it.)

The many-worlds interpretation of QM (MWI) says that the wave function does not, in fact, collapse, but instead the entire universe branches into multiples for the different possibilities described by the wave function. Bell criticized that while this is commonly presented as supposing that the wave function is "all there is", in fact it arbitrarily adds the missing discreteness:

the extended wave does not simply fail to specify one of the possibilities as actual...it fails to list the possibilities. When the M‍WI postulates the existence of many worlds in each of which the photographic plate is blackened at particular position, it adds, surreptitiously, the missing classification of possibilities. And it does so in an imprecise way, for the notion of the position of a black spot (it is not a mathematical point) [...] [or] reading of any macro‍scope instrument, is not mathematically sharp. One is given no idea of how far down towards the atomic scale the splitting of the world into branch worlds penetrates.

— J.S. Bell, "Six possible worlds of quantum mechanics", Speakable and unspeakable in quantum mechanics (anthology), 1993.

I'm inclined to agree: whatever philosophical comfort the M‍WI might provide to its adherents, it doesn't clarify the practical situation, and adds a great deal of conceptual machinery in the process of not doing so.

The transactional "interpretation" of QM is, afaik, somewhat lower-to-the-ground metaphysically. To my understanding, TI keeps everything in quantum form, and posits that spacetime events interact through a "quantum handshake": a wave propagates forward in time from an emission event, while another propagates backward in time from the corresponding absorption event, and they form a standing wave between the two while backward waves cancel out before the emission and forward waves cancel after the absorption. Proponents of the TI report that it causes the various paradoxes and conceptual anomalies of QM to disappear (cf. striking natural structure), and this makes sense to me because the "observable" Hermitian operator should be thus neatly accounted for as representing half of a quantum handshake, in which the "observer" half of the handshake is not part of the particular system under study. Wherever we choose to put the boundary of the system under study, the interface to our experimental apparatus would naturally have this half-a-handshake shape.

The practical lesson from the transactional interpretation seems to be that, for purposes of modeling QM, we don't have to worry about the wave function collapsing. If we can replicate the wave function, we're in. Likewise, if we can replicate the classical probability distributions that the wave function generates; so long as this includes all the probability distributions that result from weird quantum correlations (spooky action-at-a-distance). That the latter suffices, should be obvious since generating those probability distributions is the whole point of quantum theory; that the latter is possible is demonstrated by Bohm's hidden-variable theory (sometimes called the "Bohm Interpretation" by those focusing on its philosophy).

Genericity

There is something odd about the above list of basic elements of a QM exercise, when compared to the rewriting-calculus-inspired model we're trying to apply to it. When one thinks of a calculus term, it's a very concrete thing, with a specific representation (in fact over-specific, so that maintaining it may require α-renaming to prevent specific name choices from disrupting hygiene); and even classical physics seems to present a rather concrete representation. But the quantum distortion of the wave equation apparently applies to whatever description of a physical system we choose; to any choice of parameters and Ĥ, regardless of whether it bears any resemblance to classical physics. It certainly isn't specific to the representation of any single elementary unit, since it doesn't even blink (metaphorically) at shifting application from a one-electron to a two-electron system.

This suggests, to me anyway, two things. On the negative/cautionary side, it suggests a lack of information from which to choose a concrete representation for the "local" part of a physical system, which one might have thought would be the most straightforward and stable part of a cosmological "term". Perhaps more to the point, though, on the positive, insight-aiding side it suggests that if the quantum distortion is caused by some sort of non-local network playing out through rewrites in a dimension orthogonal to spacetime, we should consider trying to construct machinery for it that doesn't depend, much, on the particular shape of the local representation. If our distortion machinery does place some sort of constraints on local representation, they'd better be constraints that say something true about physics. Not forgetting, we expect our machinery to notice the difference between gravity and the other fundamental forces.

My most immediate goal, though, lest we forget, is to reckon whether it's at all possible any such machinery can produce the right sort of quantum distortion: a sanity check . Clues to the sort of thing one ought to look for are extremely valuable; but, having assimilated those clues, I don't atm require a full-blown theory, just a sense of what sort of thing is possible. Anything that can be left out of the demonstration probably should be. We're not even working with the best wave equation available; the Schrödinger equation is only an approximation covering the non-relativistic case. In fact, the transactional-interpretation folks tell us their equations require the relativistic treatment, so it's even conceivable the sanity check could run into difficulties because of the non-relativistic wave equation (though one might reasonably hope the sanity check wouldn't require anything so esoteric). But all this talk about relativistic and non-relativistic points out that there is, after all, something subtle about local geometry built into the form of the wave equation even though it's not directly visible in the local representation. In which case, the wave equation may still contain the essence of that co-hygienic difference between gravity and the other fundamental forces (although... for gravity even the usual special-relativistic Dirac equation might not be enough, and we'd be on to the Dirac equation for curved spacetime; let's hope we don't need that just yet).

The universe says 'hi'

Let's just pause here, take a breather and see where we are. The destination I've had my eye on, from the start of this post, was to demonstrate that a rewriting system, of the sort described, could produce some sort of quantum-like wave function. I've been lining up support, section by section, for an assault on the technical specifics of how to set up rewriting systems — and we're not ready for that yet. As noted just above, we need more information from which to choose a concrete representation. If we try to tangle with that stuff before we have enough clues from... somewhere... to guide us through it, we'll just tie ourselves in knots. This kind of exploration has to be approached softly, shifting artfully from one path to another from time to time so as not to rush into hazard on any one angle of attack. So, with spider-sense tingling —or perhaps thumbs pricking— I'll shift now to consider, instead of pieces of the cosmos, pieces of the theory.

In conventional quantum mechanics, as noted a couple of sections above, we've got basically three elements that we bring together: the parameters of our particular system of study, our classical laws of physics, and our wave equation. Well, yeah, we also have the Hermitian operator, but, as remarked earlier, we can set that aside since it's to do with interfacing to the system, which was our focus in that section but isn't what we're after now. The parameters of the particular system are what they are. The classical laws of physics are, we suppose, derived from the transformation rules of our cosmic rewriting system, with particular emphasis on the character of the primitive elements of the cosmos (whatever they are) and the geometry, and some degree of involvement of the network topology. The wave equation is also derived from the transformation rules, especially from how they interact with the network topology.

This analysis is already deviating from the traditional quantum scenario, because in the traditional scenario the classical laws of physics are strictly separate from the wave equation. We've had hints of something deep going on with the choice of wave equation; Transactional Interpretation researchers reporting that they couldn't use the non-relativistic wave equation; and then there was the odd intimation, in my recent post deriving quantum-like effects from a drastically simplified system that lacked a wave equation, that the lack of a wave equation was somehow crippling something to do with systemic coherence buried deep in the character of the mathematics. Though it does seem plausible that the wave equation would be derived more from the network topology, and perhaps the geometry, whereas the physical laws would be derived more from the character of the elementary physical components, it is perhaps only to be expected that these two components of the theory, laws and wave equation, would be coupled through their deep origins in the interaction of a single cosmological rewriting calculus.

Here is how I see the situation. We have a sort of black box, with a hand crank and input and output chutes, and the box is labeled physical laws + wave equation. We can feed into it the parameters of the particular physical system we're studying (such as a single electron in a potential field), carefully turn the crank (because we know it's a somewhat cantankerous device so that a bit of artistry is needed to keep it working smoothly), and out comes a wave function, or something akin, describing, in a predictive sense, the observable world. What's curious about this box is that we've looked inside, and even though the input and output are in terms of a classical world, inside the box it appears that there is no classical world. Odd though that is, we've gotten tolerably good at turning the crank and getting the box to work right. However, somewhere above that box, we are trying to assemble another box, with its own hand crank and input/output chutes. To this box, we mean to feed in our cosmic geometry, network topology, and transformation rules, and possibly some sort of initial classical probability distribution, and if we can get the ornery thing to work at all, we mean to turn the crank and get out of it — the physical laws plus wave equation.

Having arrived at this vision of an upper box, I was reading the other day a truthfully rather prosaic account of the party line on quantum mechanics (a 2004 book, not at all without merit as a big-picture description of mainstream thought, called Symmetry and the Beautiful Universe ), and encountered a familiar rhetorical question of such treatments: when considering a quantum mechanical wave function, "‍[...] what is doing the waving?" And unlike previous times I'd encountered that question (years or decades before), this time the answer seemed obvious. The value of the wave function is not a property of any particular particle in the system being studied, nor is it even a property of the system-of-interest as a whole; it's not part of the input we feed into the lower box at all, rather it's a property of the state of the system and so part of the output. The wave equation describes what happens when the system-of-interest is placed into the context of a vastly, vastly larger cosmos (we're supposing it has to be staggeringly vaster than the system-of-interest in order for the trick to work right), and the whole is set to jostling about till it settles into a stable state. Evidently, the shape that the lower box gives to its output is the footprint of the surrounding cosmos. So this time when the question was asked, it seemed to me that what is waving is the universe.

The upper box

All we have to work with here are our broad guesses about the sort of rewriting system that feeds into the upper box, and the output of the lower box for some inputs. Can we deduce anything, from these clues, about the workings of the upper box?

As noted, the wave function that comes out of the lower box assigns a weight to each state of the entire system-of-interest, rather than to each part of the system. Refining that point, each weight is assigned to a complete state of the system-of-interest rather than to a separable state of a part of the system-of-interest. This suggests the weight (or, a weight) is associated with each particular possibility in the classical probability distribution that we're supposing is behind the wave equation generated by the upper box. Keep in mind, these possibilities are not possible states of the system-of-interest at a given time; they're possible states of the whole of spacetime; the shift between those two perspectives is a slippery spot to step carefully across.

A puzzler is that the weights on these different possibilities are not independent of each other; they form a coherent pattern dictated by the wave equation. Whatever classical scenario spacetime settles into, it apparently has to incorporate effective knowledge of other possible classical scenarios that it didn't settle into. Moreover, different classical scenarios for the cosmos must —eventually, when things stabilize— settle down to a weight that depends only on the state of our system-of-interest. Under the sort of structural discipline we're supposing, that correlation between scenarios is generated by any given possible spacetime jostling around between classical scenarios, and thus roaming over various possible scenarios to sample them. Evidently, the key to all of this must be the transitions between cosmic scenarios: these transitions determine how the weight changes between scenarios (whatever that weight actually is, in the underlying structure), how the approach to a stable state works (whatever exactly a stable state is), and, of course, how the classical probabilities eventually correlate with the weights. That's a lot of unknowns, but the positive insight here is that the key lever for all of it is the transitions between cosmic scenarios.

And now, perhaps, we are ready (though we weren't a couple of sections above) to consider the specifics of how to set up rewriting systems. Not, I think, at this moment; I'm saturated, which does tend to happen by the end of one of these posts; but as the next step, after these materials have gone back on the shelf for a while and had a chance to become new again. I envision practical experiments with how to assemble a rewriting system that, fed into the upper box, would cause the lower box to produce simple quantum-like systems. The technique is philosophically akin to my recent construction of a toy cosmos with just the barest skeleton of quantum-like structure, demonstrating that the most basic unclassical properties of quantum physics require almost none of the particular structure of quantum mechanics. That treatment particularly noted that the lack of a wave equation seemed especially problematic; the next step I envision would seek to understand how something like a wave equation could be induced from a rewriting system. Speculatively, from there one might study how variations of rewriting system produce different sorts of classical/quantum cosmos, and reason on toward what sort of rewriting system might produce real-world physics; a speculative goal perhaps quite different from where the investigation will lead in practice, but for the moment offering a plausible destination to make sail for.

Doctor: Don't tell me you're lost too.
Shardovan: No, but as you guessed, Doctor, we people of Castrovalva are too much part of this thing you call the occlusion.
Doctor: But you do see it, the spatial anomaly.
Shardovan: With my eyes, no — but, in my philosophy.

— Doctor Who, Castrovalva , BBC.

I've made no particular secret, on this blog, that I'm looking (in an adventuresome sort of way) for alternatives to quantum theory. So far, though, I've mostly gone about it rather indirectly, fishing around the edges of the theory for possible angles of attack without ever engaging the theory on its home turf. In this post I'm going to shave things just a bit closer — fishing still, but doing so within line-of-sight of the NO FISHING sign. I'm also going to explain why I'm being so indirect, which bears on what sort of fish I think most likely here.

To remind, in previous posts I've mentioned two reasons for looking for an alternative to quantum theory. Both reasons are indirect, considering quantum theory in the larger context of other theories of physics. First, I reasoned that when a succession of theories are getting successively more complicated, this suggests some wrong assumption may be shared by all of them (here). Later I observed that quantum physics and relativity are philosophically disparate from each other (here), a disparity that has been an important motivator for TOE (Theory of Everything) physicists for decades.

The earlier post looked at a few very minor bits of math, just enough to derive Bell's Inequality, but my goal was only to point out that a certain broad strategy could, in a sense, sidestep the nondeterminism and nonlocality of quantum theory. I made no pretense of assembling a full-blown replacement for standard quantum theory based on the strategy (though some researchers are attempting to do so, I believe, under the banner of the transactional interpretation). In the later post I was even less concrete, with no equations at all.

Contents
The quantum meme
How to fish
Why to fish
Hygiene again
The structure of quantum math
The structure of reality

The quantum meme

Why fish for alternatives away from the heart of the quantum math? Aside, that is, from the fact that any answers to be found in the heart of the math already have, presumably, plenty of eyeballs looking there for them. If the answer is to be found there after all, there's no lasting harm to the field in someone looking elsewhere; indeed, those who looked elsewhere can cheerfully write off their investment knowing they played their part in covering the bases — if it was at least reasonable to cover those bases. But going into that investigation, one wants to choose an elsewhere that's a plausible place to look.

Supposing quantum theory can be successfully challenged, I suggest it's quite plausible the successful challenge might not be found by direct assault (even though eventual confrontation would presumably occur, if it were really successful). Consider Thomas Kuhn's account of how science progresses. In normal science, researchers work within a paradigm, focusing their energies on problems within the paradigm's framework and thereby making, hopefully, rapid progress on those problems because they're not distracting themselves with broader questions. Eventually, he says, this focused investigation within the paradigm highlights shortcomings of the paradigm so they become impossible to ignore, researchers have a crisis of confidence in the paradigm, and after a period of distress to those within the field, a new paradigm emerges, through the process he calls a scientific revolution. I've advocated a biological interpretation of this, in which sciences are a variety of memetic organisms, and scientific revolution is the organisms' reproductive process. But if this is so, then scientific paradigms are being selected by Darwinian evolution. What are they being selected for?

Well, the success of science hinges on paradigms being selected for how effectively they allow us to understand reality. Science is a force to be reckoned with because our paradigms have evolved to be very good at helping us understand reality. That's why the scientific species has evolved mechanisms that promote empirical testing: in the long run, if you promote empirical testing and pass that trait on to your descendants, your descendants will be more effective, and therefore thrive. So far so good.

In theory, one could imagine that eventually a paradigm would come along so consistent with physical reality, and with such explanatory power, that it would never break down and need replacing. In theory. However, there's another scenario where a paradigm could get very difficult to break down. Suppose a paradigm offers the only available way to reason about a class of situations; and within that class of situations are some "chinks in the armor", that is, some considerations whose study could lead to a breakdown of the paradigm; but the only way to apply the paradigm is to frame things in a way that prevents the practitioner from thinking of the chinks-in-the-armor. The paradigm would thus protect itself from empirical attack, not by being more explanatory, but by selectively preventing empirical questions from being asked.

What characteristics might we expect such a paradigm to have, and would they be heritable? Advanced math that appears unavoidable would seem a likely part of such a complex. If learning the subject requires indoctrination in the advanced math, then whatever that math is doing to limit your thinking will be reliably done to everyone in the field; and if any replacement paradigm can only be developed by someone who's undergone the indoctrination, that will favor passing on the trait to descendant paradigms. General relativity and quantum theory both seem to exhibit some degree of this characteristic. But while advanced math may be an enabler, it might not be enough in itself. A more directly effective measure, likely to be enabled by a suitable base of advanced math, might be to make it explicitly impossible to ask any question without first framing the question in the form prescribed by the paradigm — as quantum theory does.

This suggests to me that the mathematical details of quantum theory may be a sort of tarpit, that pulls you in and prevents you from leaving. I'm therefore trying to look at things from lots of different perspectives in the general area without ever getting quite so close as to be pulled in. Eventually I'll have to move further and further in; but the more outside ideas I've tied lines to before then, the better I'll be able to pull myself out again.

How to fish

What I'm hoping to get out of this fishing expedition is new ideas, new ways of thinking about the problem. That's ideas, plural. It's not likely the first new idea one comes up with will be the key to unlocking all the mysteries of the universe. It's not even likely that just one new idea would ever do it. One might need a lot of new ideas, many of which wouldn't actually be part of a solution — but the whole collection of them, including all the ones not finally used, helps to get a sense of the overall landscape of possibilities, which may help in turning up yet more new ideas inspired from earlier ones, and indeed may make it easier to recognize when one actually does strike on some combination of ideas that produce a useful theory.

Hence my remark, in an aside in an earlier post, that I'm okay with absurd as long as it's different and shakes up my thinking.

Case in point. In the early 1500s, there was this highly arrogant and abrasive iconoclastic fellow who styled himself Philippus Aureolus Theophrastus Bombastus von Hohenheim; ostensibly our word "bombastic" comes from his name. He rejected the prevailing medical paradigm of his day, which was based on ancient texts, and asserted his superiority to the then-highly-respected ancient physician Celsus by calling himself "Paracelsus", which is the name you've probably heard of him under. He also shook up alchemical theory; but I mention him here for his medical ideas. Having rejected the prevailing paradigm, he was rather in the market for alternatives. He advocated observing nature, an idea that really began to take off after he shook things up. He advocated keeping wounds clean instead of applying cow dung to them, which seems a good idea. He proposed that disease is caused by some external agent getting into the body, rather than by an imbalance of humours, which sounds rather clever of him. But I'm particularly interested that he also, grasping for alternatives to the prevailing paradigm, borrowed from folk medicine the principle of like affects like. Admittedly, you couldn't do much worse than some of the prevailing practices of the day. But I'm fascinated by his latching on to like-effects-like, because it demonstrates how bits of replicative material may be pulled in from almost anywhere when trying to form a new paradigm. Having seen that, it figured later into my ideas on memetic organisms.

It also, along the way, flags out the existence of a really radically different way of picturing the structure of reality. Like-affects-like is a wildly different way of thinking, and therefore ought to be a great limbering-up exercise.

In fact, like-affects-like is, I gather, the principle underlying the anthropological phenomenon of magic — sympathetic magic, it's called. I somewhat recall an anthropologist expounding at length (alas, I wish I could remember where) that anthropologically this can be understood as the principle underlying all magic. So I got to thinking, what sort of mathematical framework might one use for this sort of thing? I haven't resolved a specific answer for the math framework, yet; but I've tried to at least set my thoughts in order.

What I'm interested in here is the mathematical and thus scientific utility of the like-affects-like principle, not its manifestation in the anthropological phenomenon of magic (as Richard Cavendish observed, "The religious impulse is to worship, the scientific to explain, the magical to dominate and command"). Yet the term "like affects like" is both awkward and vague; so I use the term sympathy for discussing it from a mathematical or scientific perspective.

How might a rigorous model of this work, structurally? Taking a stab at it, one might have objects, each capable of taking on characteristics with a potentially complex structure, and patterns which can arise in the characteristics of the objects. Interactions between the objects occur when the objects share a pattern. The characteristics of objects might be dispensed with entirely, retaining only the patterns, provided one specifies the structure of the range of possible patterns (perhaps a lattice of patterns?). There may be a notion of degrees of similarity of patterns, giving rise to varying degrees of interaction. This raises the question of whether one ought to treat similar patterns as sharing some sort of higher-level pattern and themselves interacting sympathetically. More radically, one might ask whether an object is merely an intersection of patterns, in which case one might aspire to — in some sense — dispense with the objects entirely, and have only a sort of web of patterns. Evidently, the whole thing hinges on figuring out what patterns are and how they relate to each other, then setting up interactions on that basis.

I distinguish between three types of sympathy:

• Pseudo-sympathy (type 0). The phenomenon can be understood without recourse to the sympathetic principle, but it may be convenient to use sympathy as a way of modeling it.

• Weak sympathy (type 1). The phenomenon may in theory arise from a non-sympathetic reality, but in practice there's no way to understand it without recourse to sympathy.

• Strong sympathy (type 2). The phenomenon cannot, even in theory, arise from a non-sympathetic reality.

All of which gives, at least, a lower bound on how far outside the box one might think. One doesn't have to apply the sympathetic principle in a theory, in order to benefit from the reminder to keep one's thinking limber.

(It is, btw, entirely possible to imagine a metric space of patterns, in which degree of similarity between patterns becomes distance between patterns, and one slides back into a geometrical model after all. To whatever extent the merit of the sympathetic model is in its different way of thinking, to that extent one ought to avoid setting up a metric space of patterns, as such.)

Why to fish

Asking questions is, broadly speaking, good. A line comes to mind from James Gleick's biography of Feynman (quoted favorably by Freeman Dyson): "He believed in the primacy of doubt, not as a blemish upon our ability to know but as the essence of knowing." Nevertheless, one does have to pick and choose which questions are worth spending most effort on; as mentioned above, the narrow focus of normal scientific research enables its often-rapid progress. I've been grounding my questions about quantum mechanics in observations about the character of the theory in relation to other theories of physics.

By contrast, one could choose to ground one's questions in reasoning about what sort of features reality can plausibly have. Einstein did this when maintaining that the quantum theory was an incomplete theory of the physical world — that it was missing some piece of reality. An example he cited is the Schrödinger's cat thought-experiment: Until observed, a quantum system can exist in a superposition of states. So, set up an experiment in which a quantum event is magnified into a macroscopic event — through a detector, the outcome of the quantum event causes a device to either kill or not kill a cat. Put the whole experimental apparatus, including the cat, in a box and close it so the outcome cannot be observed. Until you open the box, the cat is in a superposition of states, both alive and dead. Einstein reasoned that since the quantum theory alone would lead to this conclusion, there must be something more to reality that would disallow this superposition of cat.

The trouble with using this sort of reasoning to justify a line of research is, all it takes to undermine the justification is to say there's no reason reality can't be that strange.

Hence my preference for motivations based on the character of the theory, rather than the plausibility of the reality it depicts. My reasoning is still subjective — which is fine, since I'm motivating asking a question, not accepting an answer — but at least the reasoning is then not based on intuition about the nature of reality. Intuition specifically about physical reality could be right, of course, but has gotten a bad reputation — as part of the necessary process by which the quantum paradigm has secured its ecological niche — so it's better in this case to base intuition on some other criterion.

Hygiene again

To make sure I'm fully girded for battle — this is rough stuff, one can't be too well armed for it — I want to revisit some ideas I collected in earlier blog posts, and squeeze just a bit more out of them than I did before.

My previous thought relating explicitly to Theories of Everything was that, drawing an analogy with vau-calculi, spacetime geometry should perhaps be viewed not as a playing field on which all action occurs, but rather as a hygiene condition on the interactions that make up the universe. This analogy can be refined further. The role of variables in vau-calculi is coordinating causal connections between distant parts of the term. There are four kinds of variables, but unboundedly many actual variables of each kind; and α-renaming keeps these actual variables from bleeding into each other. A particular variable, though we may think of it as a very simple thing — a syntactic atom, in fact — is perhaps better understood as a distributed, complex-structured entity woven throughout the fabric of a branch of the term's syntax tree, humming with the dynamically maintained hygiene condition that keeps it separate from other variables. It may impinge on a large part of the α-renaming infrastructure, but most of its complex distributed structure is separate from the hygiene condition. The information content of the term is largely made up of these complex, distributed entities, with various local syntactic details decorating the syntax tree and regulating the rewriting actions that shape the evolution of the term. Various rewriting actions cause propagation across one (or perhaps more than one) of these distributed entities — and it doesn't actually matter how many rewriting steps are involved in this propagation, as for example even the substitution operations could be handled by gradually distributing information across a branch of the syntax tree via some sort of "sinking" structure, mirror to the binding structures that "rise" through the tree.

Projecting some of this, cautiously, through the analogy to physics, we find ourselves envisioning a structure of reality in which spacetime is a hygiene condition on interwoven, sprawling complex entities that impinge on spacetime but are not "inside" it; whose distinctness from each other is maintained by the hygiene condition; and whose evolution we expect to describe by actions in a dimension orthogonal to spacetime. The last part of which is interestingly suggestive of my other previous post on physics, where I noted, with mathematical details sufficient to make the point, that while quantum physics is evidently nondeterministic and nonlocal as judged relative to the time dimension, one can recover determinism and locality relative to an orthogonal dimension of "meta-time" across which spacetime evolves.

One might well ask why this hygiene condition in physics should take the form of a spacetime geometry that, at least at an intermediate scale, approximates a Euclidean geometry of three space and one time dimension. I have a thought on this, drawing from another of my irons in the fire; enough, perhaps, to move thinking forward on the question. This 3+1 dimension structure is apparently that of quaternions. And quaternions are, so at least I suspect (I've been working on a blog post exploring this point), the essence of rotation. So perhaps we should think of our hygiene condition as some sort of rotational constraint, and the structure of spacetime follows from that.

I also touched on Theories of Everything in a recent post while exploring the notion that nature is neither discrete nor continuous but something between (here). If there is a balance going on between discrete and continuous facets of physical worldview, apparently the introduction of discrete elementary particles is not, in itself, enough discreteness to counterbalance the continuous feature provided by the wave functions of these particles, and the additional feature of wave-function collapse or the like is needed to even things out. One might ask whether the additional discreteness associated with wave-function collapse could be obviated by backing off somewhat on the continuous side. The uncertainty principle already suggests that the classical view of particles in continuous spacetime — which underlies the continuous wave function (more about that below) — is an over-specification; the need for additional balancing discreteness might be another consequence of the same over-specification.

Interestingly, variables in λ-like calculi are also over-specified: that's why there's a need for α-renaming in the first place, because the particular name chosen for a variable is arbitrary as long as it maintains its identity relative to other variables in the term. And α-renaming is the hygiene device analogized to geometry in physics. Raising the prospect that to eliminate this over-specification might also eliminate the analogy, or make it much harder to pin down. There is, of course, Curry's combinatorial calculus which has no variables at all; personally I find Church's variable-using approach easier to read. Tracing that through the analogy, one might conjecture the possibility of constructing a Theory of Everything that didn't need the awkward additional discreteness, by eliminating the distributed entities whose separateness from each other is maintained by the geometrical hygiene condition, thus eliminating the geometry itself in the process. Following the analogy, one would expect this alternative description of physical reality to be harder to understand than conventional physics. Frankly I have no trouble believing that a physics without geometry would be harder to understand.

The idea that quantum theory as a model of reality might suffer from having had too much put into it, does offer a curious counterpoint to Einstein's suggestion that quantum theory is missing some essential piece of reality.

The structure of quantum math

The structure of the math of quantum theory is actually pretty simple... if you stand back far enough. Start with a physical system. This is a small piece of reality that we are choosing to study. Classically, it's a finite set of elementary things described by a set of parameters. Hamilton (yes, that's the same guy who discovered quaternions) proposed to describe the whole behavior of such a system by a single function, since called a Hamiltonian function, which acts on the parameters describing the instantaneous state of the system together with parameters describing the abstract momentum of each state parameter (essentially, how the parameters change with respect to time). So the Hamiltonian is basically an embodiment of the whole classical dynamics of the system, treated as a lump rather than being broken into separate descriptions of the individual parts of the system. Since quantum theory doesn't "do" separate parts, instead expecting everything to affect everything else, it figures the Hamiltonian approach would be particularly compatible with the quantum worldview. Nevertheless, in the classical case it's still possible to consider the parts separately. For a system with a bunch of parts, the number of parameters to the Hamiltonian will be quite large (typically, at least six times the number of parts — three coordinates for position and three for momentum of each part).

Now, the quantum state of the system is described by a vector over a complex Hilbert space of, typically, infinite dimension. Wait, what? Yes, that's an infinite number of complex numbers. In fact, it might be an uncountably infinite number of complex numbers. Before you completely freak out over this, it's only fair to point out that if you have a real-valued field over three-dimensional space, that's an uncountably infinite number of real numbers (the number of locations in three-space being uncountably infinite). Still, the very fact that you're putting this thing in a Hilbert space, which is to say you're not asking for any particular kind of simple structure relating the different quantities, such as a three-dimensional Euclidean continuum, is kind of alarming. Rather than a smooth geometric structure, this is a deliberately disorganized mess, and honestly I don't think it's unfair to wish there were some more coherent reality "underneath" that gives rise to this infinite structure. Indeed, one might suspect this is a major motive for wanting a hidden variable theory — not wishing for determinism, or wishing for locality, but just wishing for a simpler model of what's going on. David Bohm's hidden variable theory, although it did show one could recover determinism with actual classical particles "underneath", did so without simplifying the mathematics — the mathematical structure of the quantum state was still there, just given a makeover as a potential field. In my earlier account of this bit of history, I noted that Einstein, seeing Bohm's theory, remarked, "This is not at all what I had in mind." I implied that Einstein didn't like Bohm's theory because it was nonlocal; but one might also object that Bohm's theory doesn't offer a simpler underlying reality, rather a more complicated one.

The elements of the vector over Hilbert space are observable classical states of the system; so this vector is indexed by, essentially, the sets of possible inputs to the Hamiltonian. One can see how, step by step, we've ended up with a staggering level of complexity in our description, which we cope with by (ironically) not looking at it. By which I mean, we represent this vast amorphous expanse of information by a single letter (such as ψ), to be manipulated as if it were a single entity using operations that perform some regimented, impersonal operation on all its components that doesn't in general require it to have any overall shape. I don't by any means deride such treatments, which recover some order out of the chaos; but it's certainly not reassuring to realize how much lack of structure is hidden beneath such neat-looking formulae as the Schrödinger equation. And the amorphism beneath the elegant equations also makes it hard to imagine an alternative when looking at the specifics of the math (as suspected based on biological assessment of the evolution of physics).

The quantum situation gets its structure, and its dynamics, from the Hamiltonian, that single creature embodying the whole of the rules of classical behavior for the system. The Schrödinger equation (or whatever alternative plays its role) governs the evolution of the quantum state vector over time, and contains within it a differential operator based on the classical Hamiltonian function.

iℏ ∂ Ψ

∂t

= Ĥ Ψ .

One really wants to stop and admire this equation. It's a linear partial differential equation, which is wonderful; nonlinearity is what gives rise to chaos in the technical sense, and one would certainly rather deal with a linear system. Unfortunately, the equation only describes the evolution of the system so long as it remains a purely quantum system; the moment you open the box to see whether the cat is dead, this wave function collapses into observation of one of the classical states indexing the quantum state vector, with (to paint in broad strokes) the amplitudes of the complex numbers in the vector determining the probability distribution of observed classical states.

It also satisfies James Clerk Maxwell's General Maxim of Physical Science, which says (as recounted by Ludwik Silberstein) that when we take the derivatives of our system with respect to time, we should end up with expressions that do not themselves explicitly involve time. When this is so, the system is "complete", or, "undisturbed". (The idea here is that if the rules governing the system change over time, it's because the system is being affected by some other factor that is varying over time.)

The equation is, indeed, highly seductive. Although I'm frankly on guard against it, yet here I am, being drawn into making remarks on its properties. Back to the question of structure. This equation effectively segregates the mathematical description of the system into a classical part that drives the dynamics (the Hamiltonian), and a quantum part that smears everything together (the quantum state vector). The wave function Ψ, described by the equation, is the adapter used to plug these two disparate elements together. The moment you start contemplating the equation, this manner of segregating the description starts to seem inevitable. So, having observed these basic elements of the quantum math, let us step back again before we get stuck.

The key structural feature of the quantum description, in contrast to classical physics, is that the parts can't be considered separately. This classical separability produced the sense of simplicity that, I speculated above, could be an ulterior motive for hidden variable theories. The term for this is superposition of states, i.e., a quantum state that could collapse into any of multiple classical states, and therefore must contain all of those classical states in its description.

A different view of this is offered by so-called quantum logic. The idea here (notably embraced by physicist David Finkelstein, who I've mentioned in an earlier post because he was lead author of some papers in the 1960s on quaternion quantum theory) is that quantum theory is a logic of propositions about the physical world, differing fundamentally from classical propositional logic because of the existence of superposition as a propositional principle. There's a counterargument that this isn't really a "logic", because it doesn't describe reasoning as such, just the behavior of classical observations when applied as a filter to quantum systems; and indeed one can see that something of the sort is happening in the Schrödinger equation, above — but that would be pulling us back into the detailed math. Quantum logic, whatever it doesn't apply to, does apply to observational propositions under the regime of quantum mechanics, while remaining gratifyingly abstracted from the detailed quantum math.

Formally, in classical logic we have the distributive law

P and (Q or R) = (P and Q) or (P and R) ;

but in quantum logic, (Q or R) is superpositional in nature, saying that we can eliminate options that are neither, yet allowing more than the union of situations where one holds and situations where the other holds; and this causes the distributive law to fail. If we know P, and we know that either Q or R (but we may be fundamentally unable to determine which), this is not the same as knowing that either both P and Q, or both P and R. We aren't allowed to refactor our proposition so as to treat Q separately from R, without changing the nature of our knowledge.

[note: I've fixed the distributive law, above, which I botched and didn't even notice till, thankfully, a reader pointed it out to me. Doh!]

One can see in this broadly the reason why, when we shift from classical physics to quantum physics, we lose our ability to consider the underlying system as made up of elementary things. In considering each classical elementary thing, we summed up the influences on that thing from each of the other elementary things, and this sum was a small tidy set of parameters describing that one thing alone. The essence of quantum logic is that we can no longer refactor the system in order to take this sum; the one elementary thing we want to consider now has a unique relationship with each of the other elementary things in the system.

Put that way, it seems that the one elementary thing we want to consider would actually have a close personal relationship with each other elementary thing in the universe. A very large Rolodex indeed. One might object that most of those elementary things in the universe are not part of the system we are considering — but what if that's what we're doing wrong? Sometimes, a whole can be naturally decomposable into parts in one way, but when you try to decompose it into parts in a different way you end up with a complicated mess because all of your "parts" are interacting with each other. I suggested, back in my first blog post on physics, that there might be some wrong assumption shared by both classical and quantum physics; well, the idea that the universe is made up of elementary particles (or quanta, whatever you prefer to call them) is something shared by both theories. The quantum math (Schrödinger equation again, above) has this classical decomposition built into its structure, pushing us to perceive the subsequent quantum weirdness as intrinsic to reality, or perhaps intrinsic to our observation of reality — but what if it's rather intrinsic to that particular way of slicing off a piece of the universe for consideration?

The quantum folks have been insisting for years that quantum reality seems strange only because we're imposing our intuitions from the macroscopic world onto the quantum-scale world where it doesn't apply. Okay... Our notion that the universe is made up of individual things is certainly based on our macroscopic experience. What if it breaks down sooner than we thought — what if, instead of pushing the idea of individual things down to a smaller and smaller scale until they sizzle apart into a complex Hilbert space, we should instead have concluded that individual things are something of an illusion even at macroscopic scales?

The structure of reality

One likely objection is that no matter how you split up reality, you'd still have to observe it classically and the usual machinery of quantum mechanics would apply just the same. There are at least a couple of ways — two come to mind atm — for some differently shaped 'slice' of reality to elude the quantum machinery.

• The alternative slice might not be something directly observable.

  Here an extreme example comes in handy (as hoped). Recall the sympathetic hypothesis, above. A pattern would not be subject to direct observation, any more than a Platonic ideal like "table" or "triangle" would be. (Actually, it seems possible a pattern would be a Platonic ideal.)

  This is also reminiscent of the analogy with vau-calculus. I noted above that much of the substance of a calculus term is made up of variables, where by a variable I meant the entire dynamically interacting web delineated by a variable binding construct and all its matching variable instances. A variable in this sense isn't, so to speak, observable; one can observe a particular instance of a variable, but a variable instance is just an atom, and not particularly interesting.

• The alternative slice might be something quantum math can't practically cope with. Quantum math is very difficult to apply in practice; some simple systems can be solved, but others are intractable. (It's fashionable in some circles to assume more powerful computers will solve all math problems. I'm reminded of a quote attributed to Eugene Wigner, commenting on a large quantum calculation: "It is nice to know that the computer understands the problem. But I would like to understand it, too.") It's not inconceivable that phenomena deviating from quantum predictions are "hiding in plain sight". My own instinct is that if this were so, they probably wouldn't be just on the edge of what we can cope with mathematically, but well outside that perimeter.

  This raises the possibility that quantum mechanics might be an idealized approximation, holding asymptotically in a degenerate case — in somewhat the same way that Newtonian mechanics holds approximately for macroscopic problems that don't involve very high velocities.

We have several reasons, by this point, to suspect that whatever it is we're contemplating adding to our model of reality, it's nonlocal (that is, nonlocal relative to the time dimension, as is quantum theory). On one hand, bluntly, classical physics has had its chance and not worked out; we're already conjecturing that insisting on a classical approach is what got us into the hole we're trying to get out of. On the other hand, under the analogy we're exploring with vau-calculus, we've already noted that most of the term syntax is occupied by distributed variables — which are, in a deep sense, fundamentally nonlocal. The idea of spacetime as a hygiene condition rather than a base medium seems, on the face of it, to call for some sort of nonlocality; in fact, saying reality has a substantial component that doesn't follow the contours of spacetime is evidently equivalent to saying it's nonlocal. Put that way, saying that reality can be usefully sliced in a way that defies the division into elementary particles/things is also another way of saying it's nonlocal, since when we speak of dividing reality into elementary "things", we mean, things partitioned away from each other by spacetime. So what we have here is several different views of the same sort of conjectured property of reality. Keeping in mind, multiple views of a single structure is a common and fruitful phenomenon in mathematics.

I'm inclined to doubt this nonlocality would be of the sort already present in quantum theory. Quantum nonlocality might be somehow a degenerate case of a more general principle; but, again bluntly, quantum theory too has had its chance. Moreover, it seems we may be looking for something that operates on macroscopic scales, and quantum nonlocality (entanglement) tends to break down (decohere) at these scales. This suggests the prospect of some form of robust nonlocality, in contrast to the more fragile quantum effects.

So, at this point I've got in my toolkit of ideas (not including sympathy, which seems atm quite beyond the pale, limited to the admittedly useful role of devil's advocate):

• a physical structure substantially not contained within spacetime.
  • space emergent as a hygiene condition, perhaps rotation-related.
  • robust nonlocality, with quantum nonlocality perhaps as an asymptotic degenerate case.
  • some non-spacetime dimension over which one can recover abstract determinism/locality.
• decomposition of reality into coherent "finite slices" in some way other than into elementary things in spacetime.
  • slices may be either non-observable or out of practical quantum scope.
  • the structural role of the space hygiene condition may be to keep slices distinct from each other.
  • conceivably an alternative decomposition of reality may allow some over-specified elements in classical descriptions to be dropped entirely from the theory, at unknown price to descriptive clarity.

I can't make up my mind if this is appallingly vague, or consolidating nicely. Perhaps both. At any rate, the next phase of this operation would seem likely to shift further along the scale toward identifying concrete structures that meet the broad criteria. In that regard, it is probably worth remarking that current paradigm physics already decomposes reality into nonlocal slices (though not in the sense suggested here): the types of elementary particles. The slices aren't in the spirit of the "finite" condition, as there are only (atm) seventeen of them for the whole of reality; and they may, perhaps, be too closely tied to spacetime geometry — but they are, in themselves, certainly nonlocal.

"Why is a raven like a writing desk?"
...
"No, I give it up," Alice replied. "What's the answer?"
"I haven't the slightest idea," said the Hatter.

— Alice's Adventures in Wonderland, Chapter 7, Lewis Carroll.

I'm always in the market for new models of how a system can be structured. A wider range of models helps keep your thinking limber; the more kinds of structure you know, the more material you have to draw inspiration from when looking for alternatives to a given theory.

Several years ago, in developing vau-calculi, I noticed a superficial structural similarity between the different kinds of variable substitution I'd introduced in my calculi, and the fundamental forces of nature in physics. (I mentioned this in an earlier blog post.) Such observed similarities can, of course, be white noise; but it's also possible that both seemingly unrelated systems could share some deep pattern that gives rise to the observed similarity. In the case of vau-calculi and physics, the two systems are so laughably disparate that for several years I didn't look past being bemused by it. But just recently I was revisiting my interest in physics TOEs (that's Theory of Everything, the current preferred name, last I heard, for what colloquially used to be called Grand Unified Theory, and before that, Unified Field Theory), and I got to thinking.

This will take a bit of set-up, and the payoff may be anticlimactic; but given the apparent extreme difficulty of making progress in this area at all, I'll take what I can get.

Contents
Substitution in vau calculi
Theories of Everything
Hygienic physics

Substitution in vau calculi

Traditionally in λ-calculi, all variables are bound by a single construct, λ, and manipulated by a single operation, called substitution. Substitution is used in two ways.

The major rearrangements of calculus terms take place in an action called β-rewriting, where a variable is completely eliminated by discarding its binding λ and replacing all references to it in the body of the old λ with some given argument. The part about eliminating the old λ is just a local adjustment in the structure of the term; but the replacement of references is done by β-substitution, which is not localized at a particular point in the term structure but instead broadcast across the body (an entire branch of the term's syntax tree). When you do this big β-substitution operation, you have to be careful. A naive rule for substituting argument A for variable x in body B would be "replace every reference to x in B with A". If you naively do that you'll get into trouble, because B might contain λs that bind either x, or some other variable that's referred to in A. Then, by following that naive rule you would lose track of which variable reference is really meant to refer to which λ. This sort of losing track is called bad hygiene.

To maintain hygiene during β-substitution, we apply α-renaming, which simply means that we replace the variable of a λ, and all the references to it, with some other variable that isn't being used for other purposes and so won't lead to confusion. This is a special case of the same sort of operation as β-substitution, in which all references to a variable are replaced with something else; it just happens that the something else is another variable. These two cases, β-substitution and α-renaming, are not perceived as separate functions, just separate uses of the same function — substitution.

It's possible to extend λ-calculus to encompass side-effectful behaviors — say, continuations and mutable storage — but to do so with well-behaved (technically, compatible ) rewriting rules, you need some sort of bounding construct to define the scope of the side-effect. In my construction of vau-calculus (a variant λ-calculus), I developed a general solution for bounding side-effects with a variable-binding construct that isn't λ, and operating the side-effects using a variable-substitution function different from β-substitution. (Discussed here.)

I ended up with four different kinds of variables, each with its own substitution operation — or operations. All four kinds of variables need α-renaming to maintain hygiene, though, and for the three kinds of side-effect variables, α-renaming is not a special case of operational substitution. If you count each variable type's α-renaming as a separate kind of substitution, there are a total of nine substitution functions (one both α and operational, three purely α, and five purely operational). λ-variables emerge as a peculiarly symmetric case, since they're the only type of variable whose substitutions (α and β) are commensurate.

This idea of multiple kinds of variables was not, btw, an unmixed blessing. One kind of variable — environment variables — turned out to be a lot more complicated to define than the others. Two kinds of variables (including those) each needed a second non-α substitution, falling into a sort of gray area, definitely not α-renaming but semi-related to hygiene and not altogether a full-fledged operational substitution. The most awkward part of my dissertation was the chapter in which I developed a general theory of rewriting systems with multiple kinds of variables and substitution functions — and the need to accommodate environment variables was at the heart of the awkwardness.

Theories of Everything

Theoretical physics can be incredibly complicated; but when looking for possible strategies to tackle the subject, imho the only practical way to think about it is to step back from the details and look at the Big Picture. So here's my take on it.

There are, conventionally, four fundamental forces: gravity, electromagnetism, the weak nuclear force, and the strong nuclear force. Gravity was the first of these we got any sort of handle on, about three and a half centuries ago with Isaac Newton's law of universal gravitation. Our understanding of electromagnetism dates to James Clerk Maxwell, about one and a half centuries ago. We've been aware of the weak nuclear force for less than a century, and the strong nuclear force for less than half a century.

Now, a bit more than a century ago, physics was based on a fairly simple, uniform model (due, afaics, to a guy about two and a half centuries ago, Roger Joseph Boscovich). Space had three Euclidean dimensions, changing with respect to a fourth Euclidean dimension of time; and this three-dimensional world was populated by point particles and space-filling fields. But then in the early twentieth century, physics kind of split in two. Two major theories arose, each of them with tremendous new explanatory power... but not really compatible with each other: general relativity, and quantum mechanics.

In general relativity, the geometry of space-time is curved by gravity — and gravity more-or-less is the curvature of space-time. The other forces propagate through space-time but, unlike gravity, remain separate from it. In quantum mechanics, waves of probability propagate through space, until observation (or something) causes the waves to collapse nondeterministically into an actuality (or something like an actuality); and various observable quantities are quantized, taking on only a discrete set of possible values. These two theories don't obviously have anything to do with each other, and leave gravity being treated in a qualitatively different way than the other forces.

Once gravity has become integrated with the geometry of space-time — through which all the forces, including gravity, propagate — it's rather hard to imagine achieving a more coherent view of reality by undoing the integration already achieved in order to treat gravity more like the other forces. As a straightforward alternative, various efforts have been taken to modify the geometry so as to integrate the other forces into it as well. This is made more challenging by the various discrete-valued quantities of quantum mechanics, as the geometry in general relativity is continuous. The phenomena for which these two theories were created are at opposite scales, and the two theories are therefore perceived as applying primarily at those scales: general relativity to the very large, and quantum mechanics to the very small; so in attempting to integrate the other forces into the geometry, modification of the geometry tends to be at the smallest scales. The two most recently-popular approaches to this are, to my knowledge, string theory and loop quantum gravity.

I've remarked in an earlier blog post, though, that the sequence of increasingly complex theories in physics seems to me likely symptomatic of a wrong assumption held in common by all the theories in the sequence (here). Consequently, I'm in the market for radically different ways one might structure a TOE. In that earlier post, I considered an alternative structure for physics, but I wasn't really looking at the TOE problem head-on; just observing that a certain alternative structure could, in a sense, eliminate one of the more perplexing features of quantum mechanics.

Hygienic physics

So here we have physics, with four fundamental forces, one of which (gravity) is somehow "special", more integrated with the fabric of things than the others are. And we have vau-calculus, with four kinds of variables, one of which (λ-variables) is somehow "special", more integrated with the fabric of things than the others are. Amusing, perhaps. Not, in itself, suggestive of a way to think about physics (not even an absurd one; I'm okay with absurd, if it's different and shakes up my thinking).

Take the analogy a bit further, though. All four forces propagate through space-time, but only gravity is integrated with it. All four operational substitutions entail α-renaming, but only β-substitution is commensurate with it. That's a more structural sort of analogy. Is there a TOE strategy here?

Well, each of the operational substitutions is capable of substantially altering the calculus term, but they're all mediated by α-renaming in order to maintain hygiene. There's really quite a lot more to term structure than the simple facets of it affected by α-renaming, with the quite a lot more being what the rewriting actions, with their operational substitutions, engage. There is, nonetheless, for most purposes only one α-renaming operation, which has to deal with all the different kinds of variables at once, because although each operational substitution directly engages only one kind of variable, doing it naively could compromise any of the four kinds of variables.

Projecting that through the structural analogy, we envision a TOE in which the geometry serves as a sort of "hygiene" condition on the forces, but is really only a tangential facet of the reality that the forces operate on — impinging on all the forces but only, after all, a hygiene condition rather than a venue. Gravity acts on the larger structure of reality in a way that's especially commensurate with the structure of the hygiene condition.

Suggestively, quantum mechanics, bearing on the three non-gravitational forces, is notoriously non-local in space-time; while the three kinds of non-λ variables mediate computational side-effects — which is to say, computational effects that are potentially non-local in the calculus term.

The status of gravity in the analogy suggests a weakness in the speculations of my earlier post on "metaclassical physics": my technique for addressing determinism and locality seems to divorce all forces equally from the geometrical structure of reality, not offering any immediately obvious opportunity for gravity to be any more commensurate with the geometry than any other force. I did mention above, that post wasn't specifically looking at TOEs; but still, I'm inclined to skepticism about an approach to fundamental physics that seeks to mitigate one outstanding problem and fails to suggest mitigation for others — that's kind of how we ended up with this bifurcated mess of relativity and quantum mechanics in the first place. As I also remarked in that post when discussing why I suspect something awry in theoretical physics, while you can tell you're using an unnatural structure by the progression of increasingly complicated descriptions, you can also tell you've hit on the natural structure when subsidiary problems just seem to melt away, and the description practically writes itself. Perhaps there's a way to add a hygiene condition to the metaclassical model, but I'd want to see some subsidiary problems melting.

Supposing one wants to try to construct a TOE, metaclassical or not, based on this strategy, the question that needs answering is, what is the primary structure of reality, to which the geometry serves a sort of tangential hygiene-enforcement role? For this, I note that the vau-calculus term structure is just a syntactic representation of the information needed to support the rewriting actions, mostly (though not exclusively) to support the substitutions. So, the analogous structure of reality in the TOE would be a representation of the information needed to support... mainly, the forces, and the particles associated with them. What we know about this information is, thus, mainly encapsulated in the table of elementary particles. Which one hopes would give us the wherewithal to encompass gravity — the analog to β-substitution — since it includes a mediating particle for mass: the Higgs boson.

[Note: I've further explored the rewriting/physics analogy in a later post, here.]