A Fundamental Duality in the Mathematical and Natural Sciences: From Logic to Biology

Category theory is the foundational theory that brings out the structure or architectonic of mathematics. Hence we will develop the Its & Dits duality as the warp and woof of the most basic ‘ur-category,’ the category of Sets (and functions)—which also underlies the other concrete categories of structured sets, e.g., groups, rings, modules, vector spaces, and so forth. Since the morphisms in Sets are set functions, we begin with the natural elements-and-distinctions definition of set functions.

Given two sets $X$ and $Y$, consider a binary relation $R\; \backslash subseteq\; X\; \backslash times\; Y$.

The relation $R$ is said to transmit (or preserve) elements if for all $x\; \backslash in\; X$, there is an ordered pair $(x,\; y)\; \backslash in\; R$ for some $y\; \backslash in\; Y$.

The relation $R$ is said to reflect elements if for all $y\; \backslash in\; Y$, there is an ordered pair $(x,\; y)\; \backslash in\; R$ for some $x\; \backslash in\; X$.

The relation $R$ is said to transmit (or preserve) distinctions if for any $(x,\; y)\; \backslash in\; R$ and $(x\text{'},\; y\text{'})\; \backslash in\; R$, if $x\; \backslash neq\; x\text{'}$, then $y\; \backslash neq\; y\text{'}$.

The relation $R$ is said to reflect distinctions if for any $(x,\; y)\; \backslash in\; R$ and $(x\text{'},\; y\text{'})\; \backslash in\; R$, if $y\; \backslash neq\; y\text{'}$, then $x\; \backslash neq\; x\text{'}$.

Ordinarily, we might say that a binary relation $R\; \backslash subseteq\; X\; \backslash times\; Y$ is the graph of a set function if it is defined everywhere on $X$ and is single-valued in $Y$. But being defined everywhere on $X$ is the same as transmitting elements and being single-valued in $Y$ is the same as reflecting distinctions.

The elements-and-distinctions definition of a function:

A function is a binary relation that transmits elements and reflects distinctions.

The notions of “transmits” and “reflects” give the directionality of the function. The two special types of set functions are injective functions and surjective functions. They are the functions that satisfy one of the two other conditions. That is, an injective function is one that transmits distinctions and a surjective function is one that reflects elements. In this manner, we see how the elements-and-distinctions duality provides the natural concepts to define functions in general and injections and surjections in particular.

The category theorist, F. William Lawvere, pointed out that every set function $f\; :\; X\; \backslash rightarrow\; Y$ determines a subset of the codomain $Y$, namely its image $f(X)\; \backslash subseteq\; Y$ and every set function $f\; :\; X\; \backslash rightarrow\; Y$ also determined a partition on its domain, namely $f^\{-1\}\; =\; \backslash \{f^\{-1\}(y)\backslash \}\_\{y\; \backslash in\; f(U)\}$. But unless the function was injective, $f$ would contain extra information such as the different elements of $X$ that got mapped to a $y\; \backslash in\; f(U)$, and unless a function was surjective, $f$ would contain extra information such as the elements of $Y$ that had no inverse image, i.e., the empty fibers $f^\{-1\}(y)\; =\; \backslash emptyset$. Hence in terms of set functions, a subset was given by an injection and a partition by a surjection [1] (p. 86). Furthermore in his introductory text, Lawvere analyzed the fundamental duality in everyday terms: “The point of view about maps indicated by the terms ‘naming,’ ‘listing,’ ‘exemplifying,’ and ‘parameterizing’ is to be considered as ‘opposite’ to the point of view indicated by the words ‘sorting,’ ‘stacking,’ ‘fibering,’ and ‘partitioning.’” [13] (p. 83)

Given two sets $X$ and $Y$ in Sets, the idea of the coproduct is to create the set with the maximum number of elements starting with $X$ and $Y$. Since $X$ and $Y$ may overlap, we must make two copies of the elements in the intersection. Hence the relevant operation is not the union of sets $X\; \backslash cup\; Y$ but the disjoint union $X\; \backslash sqcup\; Y$. To take the disjoint union of a set $X$ with itself, a copy $X^*\; =\; \backslash \{x^*\; :\; x\; \backslash in\; X\backslash \}$ of $X$ is made so that $X\; \backslash sqcup\; X$ can be constructed as $X\; \backslash sqcup\; X^*$. In a similar manner, if $X$ and $Y$ overlap, then $X\; \backslash sqcup\; Y\; =\; X\; \backslash sqcup\; Y^*$. Then the inclusions $X,\; Y\; \backslash subseteq\; X\; \backslash sqcup\; Y$, give the canonical injections $i\_X\; :\; X\; \backslash rightarrow\; X\; \backslash sqcup\; Y$ and $i\_Y\; :\; Y\; \backslash rightarrow\; X\; \backslash sqcup\; Y$.

The universal mapping property for the coproduct in Sets is that given any ‘cocone’ of maps $f\; :\; X\; \backslash rightarrow\; Z$ and $g\; :\; Y\; \backslash rightarrow\; Z$, there is a unique map $f\; \backslash sqcup\; g\; :\; X\; \backslash sqcup\; Y\; \backslash rightarrow\; Z$ such that $X\; \backslash xrightarrow\{i\_X\}\; X\; \backslash sqcup\; Y\; \backslash xrightarrow\{f\; \backslash sqcup\; g\}\; Z\; =\; X\; \backslash xrightarrow\{f\}\; Z$ and $Y\; \backslash xrightarrow\{i\_Y\}\; X\; \backslash sqcup\; Y\; \backslash xrightarrow\{f\; \backslash sqcup\; g\}\; Z\; =\; Y\; \backslash xrightarrow\{g\}\; Z$.

From the data $f\; :\; X\; \backslash rightarrow\; Z$ and $g\; :\; Y\; \backslash rightarrow\; Z$, we need to construct the unique factor map $X\; \backslash sqcup\; Y\; \backslash rightarrow\; Z$. The map $f$ contributes the image $f(X)$ subset of $Z$ and $g$ contributes image $g(Y)$ subset of $Z$ so we have the union $f(X)\; \backslash cup\; g(Y)\; \backslash subseteq\; Z$. To define the canonical factor map $f\; \backslash sqcup\; g\; :\; X\; \backslash sqcup\; Y\; \backslash rightarrow\; Z$, any $w\; \backslash in\; X\; \backslash sqcup\; Y$ is either in $i\_X(X)$ so $i\_X(x)\; =\; w$ or is in $i\_Y(Y)$ so $i\_Y(y)\; =\; w$ and then $w$ maps by $f\; \backslash sqcup\; g$ to $f(x)\; \backslash in\; f(X)$ or to $g(y)\; \backslash in\; g(Y)$, and $f(X)\; \backslash cup\; g(Y)\; \backslash subseteq\; Z$. Hence, in either case, we have the map $w\; \backslash mapsto\; z$ via the injection $f(X)\; \backslash cup\; g(Y)\; \backslash rightarrow\; Z$ that makes the triangles commutes. Both the canonical injections and the factor map were defined by inclusions, namely $X,\; Y\; \backslash subseteq\; X\; \backslash sqcup\; Y$ and $f(X)\; \backslash cup\; g(Y)\; \backslash subseteq\; Z$ (the join in the Boolean lattice of subsets on $Z$). Thus the canonical maps were defined by the inclusions in $\backslash wp(X\; \backslash sqcup\; Y)$ and $\backslash wp(Z)$.

Given two (non-empty) sets $X$ and $Y$ in Sets, the product in Sets is usually constructed as the maximum set of ordered pairs (possible distinctions) of elements from $X$ and $Y$, i.e., the Cartesian product $X\; \backslash times\; Y$.

The set $X$ defines a partition $\backslash pi\_X$ on $X\; \backslash times\; Y$ whose blocks are $B\_x\; =\; \backslash \{(x,\; y)\; :\; y\; \backslash in\; Y\backslash \}\; =\; \backslash \{x\backslash \}\; \backslash times\; Y$ for each $x\; \backslash in\; X$, and $Y$ defines a partition $\backslash pi\_Y$ whose blocks are $B\_y\; =\; \backslash \{(x,\; y)\; :\; x\; \backslash in\; X\backslash \}\; =\; X\; \backslash times\; \backslash \{y\backslash \}$ for each $y\; \backslash in\; Y$. Since $\backslash pi\_X,\; \backslash pi\_Y\; \backslash preceq\; \backslash mathbf\{1\}\_\{X\; \backslash times\; Y\}$, the induced surjections are the canonical projections $p\_X\; :\; X\; \backslash times\; Y\; \backslash rightarrow\; X$ and $p\_Y\; :\; X\; \backslash times\; Y\; \backslash rightarrow\; Y$.

The universal mapping property for the product in Sets is that given any ‘cone’ of maps $f\; :\; Z\; \backslash rightarrow\; X$ and $g\; :\; Z\; \backslash rightarrow\; Y$, there is a unique map $\backslash langle\; f,\; g\; \backslash rangle\; :\; Z\; \backslash rightarrow\; X\; \backslash times\; Y$ such that $Z\; \backslash xrightarrow\{\backslash langle\; f,\; g\; \backslash rangle\}\; X\; \backslash times\; Y\; \backslash xrightarrow\{p\_X\}\; X\; =\; Z\; \backslash xrightarrow\{f\}\; X$ and $Z\; \backslash xrightarrow\{\backslash langle\; f,\; g\; \backslash rangle\}\; X\; \backslash times\; Y\; \backslash xrightarrow\{p\_Y\}\; Y\; =\; Z\; \backslash xrightarrow\{g\}\; Y$.

From the data $f\; :\; Z\; \backslash rightarrow\; X$ and $g\; :\; Z\; \backslash rightarrow\; Y$, we need to construct the unique factor map $Z\; \backslash rightarrow\; X\; \backslash times\; Y$. The map $f$ contributes the inverse-image or coimage $f^\{-1\}\; =\; \backslash \{f^\{-1\}(x)\; :\; x\; \backslash in\; f(Z)\backslash \}$ partition on $Z$ and $g$ contributes the coimage $g^\{-1\}\; =\; \backslash \{g^\{-1\}(y)\; :\; y\; \backslash in\; g(Z)\backslash \}$ partition on $Z$ so we have the partition join $f^\{-1\}\; \backslash vee\; g^\{-1\}$ whose blocks have the form $f^\{-1\}(x)\; \backslash cap\; g^\{-1\}(y)$. To define the unique factor map $\backslash langle\; f,\; g\; \backslash rangle\; :\; Z\; \backslash rightarrow\; X\; \backslash times\; Y$, the discrete partition $\backslash mathbf\{1\}\_Z$ refines $f^\{-1\}\; \backslash vee\; g^\{-1\}$ so for each singleton $\backslash \{z\backslash \}$, there is a block of the form $f^\{-1\}(x)\; \backslash cap\; g^\{-1\}(y)$ and thus the factor map $\backslash langle\; f,\; g\; \backslash rangle$ takes $z\; \backslash mapsto\; (x,\; y)$ and the triangles commute. Both the canonical projections and the factor map were defined by partition refinements, namely $B\_X,\; B\_Y\; \backslash lesssim\; \backslash mathbf\{1\}\_\{X\; \backslash times\; Y\}$ and $f^\{-1\}\; \backslash vee\; g^\{-1\}\; \backslash lesssim\; \backslash mathbf\{1\}\_Z$ (the join in the lattice of partitions on $Z$). Thus the canonical maps were defined by the surjections in $\backslash Pi(X\; \backslash times\; Y)$ and $\backslash Pi(Z)$.

A similar analysis of the maps that are canonical (using the given information) being provided by the maps from the two partial orders of the dual lattices can be carried out for equalizers and coequalizers and thus for all limits and colimits [14] in Sets.

The most abstract form of the fundamental duality is the reverse-the-arrows of category theoretic duality. Given a category like Sets or any category $C$, the reversed arrows in the opposite category $Sets^\{op\}$ or $C^\{op\}$, are treated formally or abstractly. But in concrete category of Sets (or any category of structured sets), there are concrete binary relations that serve as the morphisms in the opposite category. One standard example of duality is in plane projective geometry where any proof involving points and lines yields another proof with the points and lines interchanged. Similarly, an arrow-theoretic proof in category theory yields a proof in the opposite category with reversed arrows (morphisms). But in the category of sets, what is interchanged to get the concrete morphisms that serve as the reversed arrows? It is the dual notions of elements (or Its) and distinctions (Dits) that are interchanged to give the dual of a function.

The elements-and-distinctions definition of a cofunction:

A cofunction is a binary relation that transmits distinctions and reflects elements.

It is easily seen from the structure of the definitions of functions and cofunctions that interchanging elements and distinctions has the same effect as interchanging “transmits” and “reflects,” which thus reverses the directionality of the morphism or arrow. Every function is a cofunction in the opposite direction. Cofunctions and functions are different binary relations; they overlap only in the case of isomorphisms. The reverse-the-arrows duality in category theory starts with this interchange of elements and distinctions in Sets to give the concrete category of sets and cofunctions $Sets^\{op\}$ and then it is abstracted as simply reversed-arrows in abstract categories. In the category of sets and cofunctions, the Cartesian product of sets satisfies the arrow-theoretic definition of the coproduct and the disjoint union of sets satisfies the definition of the product.

In general, category theory thus develops and uses the fundamental duality in abstract arrow-theoretic (“reverse the arrows”) terms. But it all started in the ‘ur-category’ of Sets where the arrows are set functions naturally defined in terms of elements and distinctions—and the dual cofunctions are defined by interchanging the role of elements and distinctions.

The next step in the development of the fundamental duality is to develop the quantitative versions of the dual notions of subsets and partitions. The quantitative measure of a subset $S\; \backslash subseteq\; U$ is its number of elements $|S|$. Probability theory starts with the assumptions of equiprobability of the elements in $U$, and then the probability of one draw from $U$ getting an element of event $S$ is the normalized cardinality of the set:

If the elements have the (always positive) point probabilities $p\; =\; (p\_1,\; \backslash dots,\; p\_n)$, then $\backslash Pr(S)\; =\; \backslash sum\_\{u\_i\; \backslash in\; S\}\; p\_i$.

Since probability theory is already well-developed, we turn to information theory based on the analogous quantitative version of partitions.

Information theory based on Shannon entropy can define that entropy in terms of the block probabilities of partitions, e.g., the inverse-image partitions of random variables [16]. But information theorists do not seem to have exploited the fundamental subset-partition duality. Gian-Carlo Rota made that key connection. As Gian-Carlo Rota and colleagues put it: “The lattice of partitions plays for information the role that the Boolean algebra of subsets plays for size or probability” [12] (p. 30). In his writings and lectures at MIT, Rota postulated that:

In his Fubini Lectures, he wrote that since “Probability is a measure on the Boolean algebra of events [subsets]” that gives quantitatively the “intuitive idea of the size of a set”, we may ask by “analogy” for some measure “which will capture some property that will turn out to be for [partitions] what size is to a set.” He went on to ask: “How shall we be led to such a property? We have already an inkling of what it should be: it should be a measure of information provided by a random variable. Is there a candidate for the measure of the amount of information?” [15] (p. 67) In view of the subset-partition duality in terms of elements and distinctions, we know the “candidate for the measure of the amount of information” in a partition $\backslash pi$, namely the number of distinctions $|\backslash text\{dit\}(\backslash pi)|$—as spelled out in Table 1. Again under the assumption of equiprobable points, the measure of information in $\backslash pi$ is the (normalized) cardinality of its ditset, its logical entropy:

where $\backslash text\{Pr\}(B\_j)\; =\; \backslash frac\{|B\_j|\}\{|U|\}$. Now $1\; =\; (\backslash sum\_\{j=1\}^m\; \backslash text\{Pr\}(B\_j))^2\; =\; \backslash sum\_\{j=1\}^m\; \backslash text\{Pr\}(B\_j)^2\; +\; \backslash sum\_\{j\; \backslash neq\; k\}\; \backslash text\{Pr\}(B\_j)\; \backslash text\{Pr\}(B\_k)$ for a general probability distribution $p\; =\; (p\_1,\; \backslash dots,\; p\_n)$ so $\backslash text\{Pr\}(B\_j)\; =\; \backslash sum\_\{u\_i\; \backslash in\; B\_j\}\; p\_i$, and the logical entropy in the general case is:

(where it might be noted that each pair of distinct indices is counted twice, e.g., as $\backslash text\{Pr\}(B\_1)\; \backslash text\{Pr\}(B\_2)$ and as $\backslash text\{Pr\}(B\_2)\; \backslash text\{Pr\}(B\_1)$). In terms of measure theory, $p$ is a finite probability measure on $U$, so $p\; \backslash times\; p$ is the product measure on $U\; \backslash times\; U$ and then logical entropy is the value of that measure on the ditset:

which is the distinctions version of the elements-formula $\backslash Pr(S)\; =\; p(S)$ for the probability measure $p$ on $U$. $\backslash Pr(S)$ is the one-draw probability of getting an element of $S$ and $h(\backslash pi)$ is the two-draw (with replacement) probability of getting a distinction of $\backslash pi$. Thus the founding of information theory on logical entropy [5] brings out the parallelism between probability theory and information theory provided by the fundamental duality and anticipated by Gian-Carlo Rota.

The fact that the information-theoretic notion of logical entropy can be interpreted in terms of dual notion of probability follows from the fact that the question of distinction or not in a partition can be expressed in subset terms. That is, the ditset $\backslash text\{dit\}(\backslash pi)$ and indit set (or equivalence relation) $\backslash text\{indit\}(\backslash pi)$ of a partition $\backslash pi$ are disjoint and complementary subsets in $U\; \backslash times\; U$, so the distinction question for $(u,\; u\text{'})$ in $\backslash pi$ is the existence question $(u,\; u\text{'})$ in $\backslash text\{dit\}(\backslash pi)$. And a probability distribution $p$ on $U$ also gives the product distribution $p\; \backslash times\; p$ on $U\; \backslash times\; U$, so the logical entropy $h(\backslash pi)$ of $\backslash pi$ can be expressed as the probability $h(\backslash pi)\; =\; p\; \backslash times\; p(\backslash text\{dit\}(\backslash pi))$.

Since logical entropy is a measure in the sense of measure theory, the compound notions are defined by the value of that measure on the appropriate set in $U\; \backslash times\; U$:

Venn diagrams arise from measures, e.g., typically counting measures. These relationships for logical entropy can be illustrated in the usual Venn diagram in Figure 1.

The question immediately arises of what is the relationship of logical entropy and the well-known Shannon entropy which for a partition $\backslash pi$ (e.g., the inverse-image partition of a random variable) is:

Any outcome with probability one carries no information (or ‘surprise’) so information is carried by the complements to one. The additive complement to one of $p\_i$ (i.e., the number added to $p\_i$ to get 1) is $1\; -\; p\_i$ and the multiplicative complement to one (i.e., the number multiplied by $p\_i$ to get 1) is $\backslash frac\{1\}\{p\_i\}$. The additive probabilistic average of the additive 1-complements is the logical entropy $\backslash sum\_i\; p\_i(1\; -\; p\_i)$. The multiplicative probabilistic average of the multiplicative 1-complements is the log-free version of the Shannon entropy $\backslash Pi\_i(\backslash frac\{1\}\{p\_i\})^\{p\_i\}$. The appropriate log is then taken to get the additive version of the Shannon entropy, e.g., logs to the base 2 for coding theory or natural logs for statistical mechanics.

The Shannon entropy is a quantification of information but not a measure in the sense of measure theory since it is not defined as a measure on a set. Yet Shannon defined the compound notions of Shannon entropy so that they satisfied the analogous Venn diagrams. This mystery [16] is explained by the non-linear but monotonic dit-to-bit transform of all the compound logical entropy formulas into the corresponding formulas for Shannon entropy:

so that

Since the dit-to-bit transform preserves the Venn diagram relationships, $h(\backslash pi\; \backslash vee\; \backslash sigma)\; =\; h(\backslash pi)\; +\; h(\backslash sigma)\; -\; m(\backslash pi,\; \backslash sigma)$ is transformed into the corresponding relation for the Shannon entropies.

The notion of logical entropy turns up in many fields (see [5]) including bioinformatics or genetic analysis ([17]; [18, chapter 4]). For instance, the sample data may be in the form of the number $N\_\{ij\}\; =\; \backslash \#$ of ordered $(i,\; j)$ pairs in the sample. Then the sample statistic for heterogeneity is:

If it were $N$ independent draws of ordered pairs from the probability distribution $p$, then the probability of each pair is $E(N\_\{ij\})\; =\; p\_i\; p\_j$ so the expected value of the statistic is:

Since probability and logical entropy arise as the quantitative versions of the dual notions of subsets and partitions, the notion of logical entropy gives the fundamental or logical notion of information-as-distinctions and the Shannon entropy arises as the transform that has powerful applications in what Shannon called the “A Mathematical Theory of Communication.”[19] The full argument why the notion of logical entropy provides a logical foundation for what is usually called “Information Theory” and provides the definition of information-as-distinctions has been spelled out elsewhere.([5] ; [7])

The important point for our purposes at hand is that probability theory and logical information theory (based on logical entropy) both start with the quantitative versions of the duality between subsets and partitions—based on the counting of the elements and distinctions (or Its & Dits).

Quantum mechanics (QM) has a distinctive type of mathematics, i.e., all vectors are states (which implies the superposition principle) and observables are operators, quite different from the math of classical mechanics. Our analysis is of that distinctive math of QM, not the physics. The thesis is that the math of QM is the Hilbert space version of the math of partitions, or, put the other way around, partition math is a bare-bones, schematic, or skeletal version of QM math. The notion of a superposition state is the basic notion in QM that separates it from the fully-definite or definite-all-the-way-down metaphysics of classical mechanics. When referring to a quantum particle (not the classical notion of a particle) as a “quanton,” Mario Bunge makes that point.

Another surprising peculiarity of quantons is that they are blurry or fuzzy rather than neat or sharp. Whereas in classical physics all properties are sharp, in quantum physics only a few are: most are blunt or smudged. ... The reason for this fuzziness is that ordinarily an isolated quanton is in a “coherent” state, that is, the combination or superposition (weighted sum) of two or more basic states (or eigenfunctions). The superposition or “entanglement” of states is a hallmark of quantum mechanics [27] (pp. 49-50).

If quantum field theory is also included, then James Cushing makes the same point, namely that “superposition, with the attendant riddles of entanglement and reduction, remains the central and generic interpretative problem of quantum theory” [28] (p. 34).

Our thesis about QM math provides the logical basis to interpret superposition in terms of indefiniteness since partitions provide the logical model of the indefiniteness of the states in a non-singleton block of a partition, i.e., in a non-singleton equivalence class in an equivalence relation. Given a superposition state $\backslash alpha|a\backslash rangle\; +\; \backslash beta|b\backslash rangle$, the support (forget the vector space machinery) is the set $\backslash \{a,\; b\backslash \}$, so the schematic set-version of a superposition state is its support (as a non-singleton equivalence class or block in a partition). This thesis has been argued at length in papers ([29], [30]) and a book [8]. Hence we will only summarize some of the salient points here.

We will demonstrate the thesis by briefly describing the partition math version of quantum states, quantum observables, and quantum state reduction (‘measurement’). The mathematical tool that brings out the partition aspects of quantum states is not the state vector representation but the density matrix representation. Hence we construct the density matrix version of a partition $\backslash pi\; =\; \backslash \{B\_1,\; \backslash dots,\; B\_m\backslash \}$ on a set $U$ with positive point probabilities $p\; =\; (p\_1,\; \backslash dots,\; p\_n)$. $U$ is interpreted as the set of possible eigenstates of a quantum particle (“eigen” is interpreted as “definite”). For each block $B\_j$, let $|b\_j\backslash rangle$ be the $n$-ary real column vector with the $i^\{\backslash text\{th\}\}$ entry being $\backslash sqrt\{\backslash frac\{p\_i\}\{\backslash Pr(B\_j)\}\}$ if $u\_i\; \backslash in\; B\_j$ and 0 otherwise. These vectors are normalized and, since the blocks are disjoint, the vectors are orthogonal to each other so $\backslash langle\; b\_k\; |\; b\_j\; \backslash rangle\; =\; \backslash delta\_\{jk\}$ (the Kronecker delta where $\backslash delta\_\{jk\}\; =\; 1$ if $j\; =\; k$ and 0 otherwise). Then the $n\; \backslash times\; n$ density matrix $\backslash rho(B\_j)$ is constructed as the outer product of $|b\_j\backslash rangle$ with its transpose $|b\_j\backslash rangle^t\; =\; \backslash langle\; b\_j|$:

The entries in $\backslash rho(B\_j)$ are $\backslash rho(B\_j)\_\{ik\}\; =\; \backslash frac\{\backslash sqrt\{p\_i\; p\_k\}\}\{\backslash Pr(B\_j)\}$ if $u\_i,\; u\_k\; \backslash in\; B\_j$, else 0. Then the density matrix $\backslash rho(\backslash pi)$ for the partition is the probabilistic sum of the density matrices for the blocks:

The entries in $\backslash rho(\backslash pi)$ are $\backslash rho(\backslash pi)\_\{ik\}\; =\; \backslash sqrt\{p\_i\; p\_k\}$ if $(u\_i,\; u\_k)\; \backslash in\; \backslash text\{indit\}(\backslash pi)$, else 0, so the non-zero entries of $\backslash rho(\backslash pi)$ correspond to the ordered pairs in the equivalence relation $\backslash text\{indit\}(\backslash pi)$ and the zeros correspond to the ordered pairs in $\backslash text\{dit\}(\backslash pi)$. If $\backslash rho(\backslash pi)\_\{ik\}\; >\; 0$ ($i\; \backslash neq\; k$), then the $u\_i$ and $u\_k$ are blurred or cohered together in one ‘superposition’ block. Those non-zero off-diagonal elements, indicating the presence of superposition in the corresponding diagonal elements, are called “coherences” in QM and they allow the characteristic interference effects.

For this reason, the off-diagonal terms of a density matrix ... are often called “quantum coherences” because they are responsible for the interference effects typical of quantum mechanics that are absent in classical dynamics [31] (p. 177).

A density matrix $\backslash rho$ represents a pure state if $\backslash rho^2\; =\; \backslash rho$, otherwise a mixed state. All the $\backslash rho(B\_j)$ are pure states and the only partition with a pure state density matrix is $\backslash mathbf\{0\}\_U$. Any density matrix is positive Hermitian so its $n$ eigenvalues are non-negative reals and sum to one. In the case of $\backslash rho(\backslash pi)$, the eigenvalues are the $m$ block probabilities $\backslash Pr(B\_j)$ and $n\; -\; m$ zeros. In the case of a pure density matrix such as

$\backslash rho(\backslash mathbf\{0\}\_U)$ or $\backslash rho(B\_j)$, there is one eigenvalue of 1 with the rest of the eigenvalues of zero. Given any $\backslash rho(B\_j)$ [$\backslash rho(\backslash mathbf\{0\}\_U)$ being the special case where $B\_1\; =\; U$], the vector $|b\_j\backslash rangle$ is recovered (up to sign) as the normalized eigenvector associated with the eigenvalue of 1, and $\backslash rho(B\_j)\; =\; |b\_j\backslash rangle\backslash langle\; b\_j|$ follows as the spectral decomposition of the density matrix.

Taking $S\; =\; B\_j$, a pure state density matrix $\backslash rho(S)$ for a subset $S\; \backslash subseteq\; U$ has the normalized eigenvector $|s\backslash rangle$ associated with the eigenvalue of 1. The probability of drawing $u\_i$ given $S$ is given by the formula: $\backslash Pr(u\_i|S)\; =\; \backslash langle\; u\_i|s\backslash rangle^2$—which shows the origin of the Born Rule at the set level. Hence that vector $|s\backslash rangle$ plays the role of the state vector or (non-wavy) ‘wave function.’ at the set level.

These properties of partition math formulated using the density matrices $\backslash rho(\backslash pi)$ of partitions all hold in the Hilbert space math of QM. Those corresponding properties are summarized in Table 2.

There is (in the mathematical folklore) a semi-algorithmic procedure to associate vector space concepts with the corresponding set concepts. For instance, a subspace is the vector space concept that corresponds to the set concept of a subset. We call this procedure, the:

Yoga of Linearization.

Given a basis set $U$ of a vector space, consider it first as just a set, apply a set concept to the set $U$,
and then take the vector space notion linearly generated by it
as the corresponding vector space concept.

The Yoga of Linearization can be viewed as an embellishment on the free vector space functor from the category of Sets to the category of vector spaces over a given field, i.e., $\backslash mathbb\{C\}$ for our application to QM. A subset $S$ generates a subspace $[S]$ and the cardinality of the subset $|S|$ corresponds to the dimension $\backslash dim([S])$ of the subspace. Given a partition $\backslash pi$ on $U$ as a set, each block $B\_j$ generates a subspace $[B\_j]$ and the collection $\backslash \{[B\_j]\backslash \}\_\{j=1\}^m$ constitutes a direct-sum decomposition (DSD) of the vector space where a DSD of a vector space is a set of subspaces so that each non-zero vector in the space can be uniquely represented as a sum of (non-zero) vectors from the subspaces. In particular, those vectors in the sum are the non-zero projections of the vector to the subspaces.

Thus we may say that the vector space version of a set partition is a DSD. Moreover, we could have defined a partition $\backslash pi$ on $U$ as a set of subsets $\backslash \{B\_j\backslash \}\_\{j=1\}^m$ so that each non-empty subset of $U$ can be uniquely represented as the union of non-empty subsets of the $B\_j$s. If the union of the $B\_j$s was not all of $U$, then the difference $U\; -\; \backslash cup\_\{j=1\}^m\; B\_j$ would have no representation as a union of non-empty subsets of the $B\_j$s, and if $B\_j\; \backslash cap\; B\_k\; \backslash neq\; \backslash emptyset$, then that overlap would have two representations.

An observable is a Hermitian (or self-adjoint) operator on a Hilbert space $F\; :\; V\; \backslash rightarrow\; V$ which will have real eigenvalues. The set version is a numerical attribute $f\; :\; U\; \backslash rightarrow\; \backslash mathbb\{R\}$ where $U$ is a basis set for $F$. Given any numerical attribute $f\; :\; U\; \backslash rightarrow\; \backslash mathbb\{R\}$, a Hermitian operator $F$ is defined on $V$ by the definition $Fu\_i\; =\; f(u\_i)u\_i$ (or $F|u\_i\backslash rangle\; =\; f(u\_i)|u\_i\backslash rangle$ if we use the Dirac notation) on the basis $U$ and then linearly extended to the whole space. Or given a Hermitian operator $F\; :\; V\; \backslash rightarrow\; V$ and an orthonormal basis $U$ of eigenvectors of $F$, the numerical attribute is recovered as the eigenvalue function $f\; :\; U\; \backslash rightarrow\; \backslash mathbb\{R\}$ that assigns to each eigenvector its eigenvalue. The numerical attribute $f\; :\; U\; \backslash rightarrow\; \backslash mathbb\{R\}$ has the inverse-image partition $f^\{-1\}\; =\; \backslash \{f^\{-1\}(r)\backslash \}\_\{r\; \backslash in\; f(U)\}$ and the eigenspaces for the $F$ defined by $f$ are the subspaces $[f^\{-1\}(r)]$ generated by the blocks $f^\{-1\}(r)$ for the eigenvalues $r\; \backslash in\; f(U)$.

What is the set notion of an eigenvector? For a subset $S\; \backslash subseteq\; U$ and real $r\; \backslash in\; \backslash mathbb\{R\}$, let “$rS$” stand for the statement “the value of $f$ on the subset $S$ is $r$”, so that “$f\; \backslash upharpoonright\; S\; =\; rS$” ($f\; \backslash upharpoonright\; S$ is $f$ restricted to $S$) is the set version of the eigenvalue equation: $F|u\_i\backslash rangle\; =\; r|u\_i\backslash rangle$. Thus the set notion of an eigenvector is just a constant set of a numerical attribute and its eigenvalue is that constant value on the set. A characteristic or indicator function $\backslash chi\_S\; :\; U\; \backslash rightarrow\; \backslash \{0,\; 1\backslash \}\; \backslash subseteq\; \backslash mathbb\{R\}$, where $\backslash chi\_S(u\_i)\; =\; 1$ if $u\_i\; \backslash in\; S$, else 0, defines the projection operator $P\_\{[S]\}\; :\; V\; \backslash rightarrow\; V$ to the subspace generated by the subset $S$. Thus characteristic functions on sets correlate with projection operators on vector spaces. Moreover, each observable $F$ with the eigenvalues $\backslash lambda\_1,\; \backslash dots,\; \backslash lambda\_n$ and eigenspaces $\backslash \{V\_\{\backslash lambda\_i\}\backslash \}$ has a spectral decomposition $F\; =\; \backslash sum\_\{i=1\}^n\; \backslash lambda\_i\; P\_\{V\_i\}$. Hence the corresponding spectral decomposition of a numerical attribute $f\; :\; U\; \backslash rightarrow\; \backslash mathbb\{R\}$ is $f\; =\; \backslash sum\_\{r\; \backslash in\; f(U)\}\; r\; \backslash chi\_\{f^\{-1\}(r)\}\; :\; U\; \backslash rightarrow\; \backslash mathbb\{R\}$.

Applied to observables, our thesis that the QM math of observables is the Hilbert space version of the partition math of numerical attributes over the reals. Those correlations between the partition math of numerical attributes and QM math of observables are given in Table 3.

Given an observable $F\; :\; V\; \backslash rightarrow\; V$ with an ON (Ortho-Normal) basis of eigenvectors $U$, an eigenvalue function $f\; :\; U\; \backslash rightarrow\; \backslash mathbb\{R\}$, a DSD of eigenspaces $\backslash \{V\_r\backslash \}$ associated with the eigenvalues $r\; \backslash in\; f(U)$, the projective measurement of a state $|\backslash psi\backslash rangle$ with density matrix $\backslash rho$ is described by the Lüders mixture operation ([32]; [33]) which produces a mixed state density matrix

Hilbert space Lüders Mixture Operation

where $P\_r$ is the projection to the eigenspace $V\_r\; =\; [f^\{-1\}(r)]$. To see the set version, we start with the numerical attribute $f\; :\; U\; \backslash rightarrow\; \backslash mathbb\{R\}$ where the $n\; \backslash times\; n$ projection matrices $P\_r$ for $r\; \backslash in\; f(U)$ are diagonal matrices with the diagonal entries given by $(P\_r)\_\{ii\}\; =\; \backslash chi\_\{f^\{-1\}(r)\}(u\_i)$. Then the set version of the Lüders mixture operation on a density matrix $\backslash rho(\backslash pi)$ is given by:

Partition version of Lüders mixture operation

It is then easily shown [29] that $\backslash hat\{\backslash rho\}(\backslash pi)\; =\; \backslash rho(\backslash pi\; \backslash vee\; f^\{-1\})$. Thus the set version of the Lüders mixture operation is density matrix for the join of two partitions, $\backslash pi$ representing the state being measured, and $f^\{-1\}\; =\; \backslash \{f^\{-1\}(r)\backslash \}\_\{r\; \backslash in\; f(U)\}$ representing the observable.

These results, which only give a small part of the partition math underlying QM math [8], are summarized in Table 4.

The partition math basis for QM mathematics shows a new way to handle the century-old problem of interpreting the QM formalism. The “cutting at the joint” between QM math and QM physics is indicated by the absence of Planck’s constant in our analysis that deals only with the math of vector spaces and Hilbert spaces in particular. The all-important superposition principle (that the sum of two quantum states is another possible quantum state) and Dirac’s use of CSCOs (Complete Sets of Commuting Observable) do not involve Planck’s constant.

Partitions (or equivalence relations) are the math to model distinctions and indistinctions and thus to model indefiniteness (states of a particle in a non-singleton ‘superposition’ block of a partition or equivalence class of an equivalence relation) as opposed to definite or eigen-states (singleton blocks as in the discrete partition $1\_U$). This approach to understanding QM corroborates an interpretation by Heisenberg, Shimony, and many others who see quantum reality (like the part of an iceberg under the water) that is characterized by objective indefiniteness.

The conceptual elements of quantum theory that now underlie our picture of the physical world include objective chance, quantum interference, and the objective indefiniteness of dynamical quantities. Quantum interference, which is directly observable, was readily absorbed by the physics community. Objective chance and indefiniteness, being of more philosophical significance, gained acceptance only after much debate and conceptual analysis, when it was recognized that observed phenomena are better understood through these notions than through older ones or hidden variables [34] (p. vii).

Heisenberg, Shimony, Jaeger, and others may describe an indefinite superposition as being a “potentiality” as opposed to an actuality but that should be interpreted as a manner of speaking about indefiniteness rather than as a different ontological category. There is only one ontological category of reality but the real state may be indefinite between a number of definite or eigen-states.

The Feynman rules [34] (pp. 110-111) specify that making the change from indefinite to more definite is by making distinctions. Different levels of indefiniteness may be schematically pictured, in an anschaulich (intuitive) manner, using a lattice of partitions where a state reduction (or ‘measurement’) moves upward (‘vertically’) in the lattice from indefinite to more definite states—which von Neumann called a Type I quantum process. The Type II quantum process is a unitary transformation that moves horizontally at the same level of indefiniteness transforming one basis set $\backslash \{a,\; b,\; c\backslash \}$ into another basis set $\backslash \{a\text{'},\; b\text{'},\; c\text{'}\backslash \}$ as pictured in Figure 3. In the schematic terms of the lattice of partitions, Figure 3 shows the classical part of reality (fully definite states as the “tip of the iceberg”) and the quantum reality involving indefinite superposition states (like the “underwater part of an iceberg”).

The idea of the two basic processes in QM has worried some quantum philosophers. Classical mechanics has no superposition states, only fully definite states, and only one type of fundamental process that transforms the definite states into other definite states.

The schematic picture of Figure 3 shows how it is natural to have two fundamental processes, the vertical process of going from indefinite to more definite and the horizontal process of moving at the same level of indefiniteness. Moreover, this shows why it is natural to have only one type of fundamental process in classical mechanics. We have seen in the partition logic Principle of Identity of Indistinguishables that classicality is represented by the discrete partition $1\_U$. But at that classical level, there can be no more vertical movement from indefinite to more definite since the classical states are fully definite—so there is only the horizontal movement from definite states to other definite states.

It should also be noted that boundary between state reductions (“measurements”) and unitary evolution is specified in the Feynman rules in terms of distinguishability and indistinguishability—concepts modeled at the logical level by partitions. The Feynman rules were stated in his work in the early 1950s, e.g., [35].

The non-commuting or even conjugate operators of QM math at first seem to have little connection with partition math. But each observable operator has the associated direct-sum decomposition of eigenspaces, and DSDs are the vector space version of partitions. Suppose we have two observables $F,\; G\; :\; V\; \backslash rightarrow\; V$ with the respective DSDs of eigenspaces $\backslash \{V\_i\backslash \}\_\{i\; \backslash in\; I\}$ and $\backslash \{W\_j\backslash \}\_\{j\; \backslash in\; J\}$. We know that the operation on partitions to create more distinctions is the join so we consider a join-like operation on the two DSDs to yield the set of non-zero subspaces $\backslash \{V\_i\; \backslash cap\; W\_j\backslash \}$. Partitions on the same set (or numerical attributes on the set) are said to be compatible, and the join of two partitions on a set $U$ is always another partition on $U$. But these subspaces of simultaneous eigenvectors may not span the whole space $V$. Let $\backslash mathcal\{S\}\backslash mathcal\{E\}$ be the space spanned by the simultaneous eigenvectors in the non-zero spaces $V\_i\; \backslash cap\; W\_j$. Then it is a theorem ([29], [8]) that $F$ and $G$ commute iff $\backslash mathcal\{S\}\backslash mathcal\{E\}\; =\; V$, and $F$ and $G$ are conjugate iff $\backslash mathcal\{S\}\backslash mathcal\{E\}\; =\; \backslash \{0\backslash \}$ (the zero space), i.e., they have no simultaneous eigenvectors.

Thus commutativity depends solely on the vector-space partitions (DSDs), not on the operators per se. In vector spaces like $\backslash mathbb\{Z\}\_2^n$, the only operators are projection operators (with eigenvalues of 0 or 1), but DSDs can have up to $n$ subspaces and the DSDs can be commuting, non-commuting, or even conjugate. The join-like operation on DSDs is only properly called a join in the case of commutativity.

The Heisenberg indeterminacy principle is usually stated in a quantitative form involving Planck's constant, but the underlying fact that there are conjugate DSDs with no simultaneous eigenvectors (i.e., $\backslash mathcal\{S\}\backslash mathcal\{E\}\; =\; \backslash \{0\backslash \}$) is a fact about vector spaces that has nothing to do with Planck's constant [8].

One of the basic operations on partitions that we will see in many contexts is the join of enough partitions to reach the discrete partition, i.e., to distinguish all the elements of $U$. If the partitions are the inverse-images of numerical attributes then we have one of the word-for-word translation dictionaries between partition math and the QM math (where Planck's constant plays no role).

Set case: A set of compatible numerical attributes $f,\; g,\; \backslash dots,\; h\; :\; U\; \backslash rightarrow\; \backslash mathbb\{R\}$ is said to be complete (a Complete Set of Compatible Attributes or CSCA) if the join of their inverse-image partitions is the partition with all blocks of cardinality one.

Then each element $u\_i\; \backslash in\; U$ is uniquely specified by the ordered set of its values.

QM case: A set of commuting observables $F,\; G,\; \backslash dots,\; H\; :\; V\; \backslash rightarrow\; V$ is said to be complete (a Complete Set of Commuting Observables or CSCO [36]) if the join of their eigenspace DSDs is a DSD with all subspaces of dimension one. Then each simultaneous eigenvector is uniquely specified by the ordered set of its eigenvalues.

There is one mathematical theory, group representation theory, that is particularly applicable to quantum mechanics and particle physics. That is because a group is essentially a ‘dynamic’ way to define an equivalence relation. An equivalence relation on a set is reflexive, symmetric, and transitive. As Hermann Weyl pointed out: “The three postulates for a group simply state that each figure is similar to itself and that similarity is symmetric and transitive (see the axioms for equivalence on p. 9)” [37] (p. 73).

Given a group $G$ and a set $U$, a representation of $G$ on $U$ (or a $G$ group action on $U$) is a set of isomorphisms (i.e., permutations) $\backslash \{R\_g\; :\; U\; \backslash rightarrow\; U\backslash \}\_\{g\; \backslash in\; G\}$, such that (1) for the identity $e\; \backslash in\; G$, $R\_e$ is the identity map on $U$, (2) for any $g\; \backslash in\; G$ with its inverse $g^\{-1\}$, $R\_g\; R\_\{g^\{-1\}\}\; =\; R\_\{g^\{-1\}\}\; R\_g\; =\; R\_e$, and (3) For $g,\; g\text{'},\; g\text{'}\text{'}\; \backslash in\; G$, if $gg\text{'}\; =\; g\text{'}\text{'}$ then $R\_g\; R\_\{g\text{'}\}\; =\; R\_\{g\text{'}\text{'}\}$. When $G$ acts on $U$, then it defines a partition, the orbit partition. For any $u\; \backslash in\; U$, the orbit, block, or equivalence class containing $u$ is the set $\backslash \{u\text{'}\; \backslash in\; U\; :\; \backslash exists\; R\_g,\; R\_g(u)\; =\; u\text{'}\backslash \}$ of elements of $U$ that can be reached by the action of some $R\_g$. In other words, the actions of the group $R\_g(u)\; =\; u\text{'}$ are creating the indistinctions $(u,\; u\text{'})$ of the orbit partition. Often a group is described as a symmetry group so if $R\_g(u)\; =\; u\text{'}$ then $u$ is said to be symmetric to $u\text{'}$—so that being symmetric is the equivalence relation whose equivalence classes are the orbits.

In the development of the math of partitions, we have seen that a (non-discrete) partition can be refined by adding more distinctions, e.g., $\backslash pi$ can be refined to $\backslash pi\; \backslash vee\; \backslash sigma$ by adding the new distinctions of $\backslash sigma$ since $\backslash text\{dit\}(\backslash pi\; \backslash vee\; \backslash sigma)\; =\; \backslash text\{dit\}(\backslash pi)\; \backslash cup\; \backslash text\{dit\}(\backslash sigma)$ so the new distinctions are $\backslash text\{dit\}(\backslash sigma)\; -\; \backslash text\{dit\}(\backslash pi)$. If $H$ is a subgroup of $G$, then $\backslash \{R\_g\; :\; U\; \backslash rightarrow\; U\backslash \}\_\{g\; \backslash in\; H\}$ is a group representation of $H$ on $U$ and since it has no indistinctions $R\_g(u)\; =\; u\text{'}$ for $g\; \backslash in\; G\; -\; H$, its orbit partition will refine the orbit partition of the $G$-representation.

This way to create more distinctions is called “symmetry breaking”; it creates smaller and thus less indefinite or more definite orbits. In the lattice of partitions, the most refined partition is the discrete partition $\backslash mathbf\{1\}\_U$, and it is the orbit partition of the smallest subgroup $\backslash \{e\backslash \}$ consisting of just the identity $e$.

As would be expected from the Yoga of Linearization, the set concepts linearize to vector space representations of a group. Given a group $G$ and a vector space $V$ over $\backslash mathbb\{C\}$, a vector space representation of $G$ is a set of invertible linear maps $\backslash \{R\_g\; :\; V\; \backslash rightarrow\; V\backslash \}$ satisfying $R\_e\; =\; I$ and $R\_g\; R\_\{g\text{'}\}\; =\; R\_\{gg\text{'}\}$. These vector space representations have very important applications in quantum mechanics [38] and particle physics [39]—as would be expected since a group representation is a ‘dynamic’ way to define a partition (of orbits) in the set case and a direct-sum decomposition (of irreducible subspaces of $V$) in the vector space case. The approach to isolating the irreducible representation or irreps (representations restricted to irreducible subspaces) developed by the Nanjing School of J. Q. Chen and colleagues is particularly appropriate for our purposes since “the foundation of the new approach is precisely the theory of the complete set of commuting operators (CSCO) initiated by Dirac...” [39] (p. 2).

It is well beyond the scope of this paper to go into the other major theory of modern physics, general relativity, but suffice it to say that indefiniteness plays a key role there as well as emphasized by the general relativity theorist, John Stachel.

So both relativity and quantum theory lead to the same conclusion: Leibniz’s principle is not universally applicable. There is a category of entities with quiddity but no inherent haecceity. Given that both general relativity and quantum mechanics are based on such entities, it is difficult to believe that, in any theory purporting to underlie both relativity and quantum theory, inherent individuality would re-emerge in its fundamental entities, whatever they are... . [26] (p. 55)

The elements-and-distinctions or Its & Dits duality leads in the life sciences to two types of mechanisms, the well-known selectionist mechanism and the ‘dual’ mechanism that will be called the “generative mechanism.”

The selectionist mechanism, abstractly described, is a process that constantly whittles down sets of actual entities or elements, e.g., the set of random variations of a type of organism, to subsets that are selected according to some fitness criterion.

In contrast, a generative mechanism operates on some relatively undifferentiated entity (a root or stem) containing a number of potential outcomes so that making distinctions will generate a variety of different possible outcomes. The making of distinctions can be conceptualized as the implementation of a code or as symmetry-breaking.

The question of existence or non-existence on one side of the duality is dual to the question of distinction or indistinction on the other side.

The following Figure 4 abstractly illustrates the different mechanisms:

Since the selectionist type of mechanism is already well known and much promoted ([40]; [41]), we will focus mostly on developing the relevant concepts to describe generative mechanisms.

We live in an ‘Information Age’ so we begin by showing how the machinery of information coding embodies generative mechanisms. Mathematically, a partition on a set represents one way to differentiate the elements of the set into different blocks. The join with another partition generates a partition with more refined (smaller) blocks that makes all the distinctions of the partitions in the join. Starting from a single block consisting of the set of all possibilities like the unbranched root of the tree (symbolized by the indiscrete partition $\backslash mathbf\{0\}\_U$), a sequence of partitions joined together differentiates all the elements of the set ultimately into singleton blocks (i.e., $\backslash mathbf\{1\}\_U$) that are the leaves of the tree. All the (instantaneous) codes of coding theory can be generated in this way and then the codes are implemented in practice to traverse the tree to generate the coded outcomes (e.g., messages).

With consecutive joins of partitions (always on the same universe set), the blocks get smaller and smaller until they reach the discrete partition $\backslash mathbf\{1\}\_U$ (like in a CSCA or CSCO) with the smallest non-empty blocks being the singletons of elements of $U$. The least refined partition is the indiscrete partition $\backslash mathbf\{0\}\_U\; =\; \backslash \{U\backslash \}$ whose only block is all of $U$ and it represents the root (or stem as in stem cell) of the tree.

The tree that would illustrate the consecutive joins in Table 5 where $U\; =$

$\backslash \{a,\; b,\; c\backslash \}$ consists of three leaves or messages. Since the code is binary, all the partitions to be joined are binary with the first block on the left labeled with the code letter 0 and the other block is labeled 1 as in $\backslash \{\backslash overset\{0\}\{\backslash \{a\backslash \}\},\; \backslash overset\{1\}\{\backslash \{b,\; c\backslash \}\}\backslash \}$, i.e., it is like a numerical attribute on $U$ taking values in the set of code letters with the code letter assigned to a block being the value of the attribute on those elements of $U$. When a message first appears in the Consecutive Joins column as a singleton, then its history of 0's and 1's in the second column gives its code.

In Figure 5, the partition joins are indicated and the trajectory from the complete ‘superposition’ state $\backslash mathbf\{0\}\_U$ at the root of the tree to the messages is given in the (upside down) tree diagram with the rows of Table 5 indicated.

At each junction in the tree, there is pictured a switch which (to borrow the language from QM) reduces the superposition state to one of the two more definite outcomes. That is, the first switch at the root $\backslash mathbf\{0\}\_U\; =\; \backslash \{\backslash \{a,\; b,\; c\backslash \}\backslash \}$ reduces it to one of the more definite states in $\backslash \{\backslash \{a\backslash \},\; \backslash \{b,\; c\backslash \}\backslash \}$ and then the second switch reduces the superposition $\backslash \{b,\; c\backslash \}$ to $\backslash \{b\backslash \}$ or $\backslash \{c\backslash \}$. The final result is the fully definite states of $\backslash mathbf\{1\}\_U$, i.e., the leaves in the code tree.

For a more complex example, consider the five messages in $U\; =\; \backslash \{u\_1,\; \backslash dots,\; u\_5\backslash \}$. To generate a binary code for the five outcomes we consider the repeated joins of binary partitions in Table 6. Think of the block on the left as representing the code letter 0 and the block on the right as representing the code letter 1. In the repeated joins of binary partitions, the blocks get smaller and smaller until a singleton block is reached for each message—as we saw before in CSCAs and CSCOs. When a message first appears as a singleton (i.e., fully differentiated outcome) in the Consecutive Joins column representing the sequence of more and more refined partitions, then the sequence of 0-blocks or 1-blocks in the Partitions column containing that specific outcome give the code for that outcome or message [42] (p. 56).

For instance, the $u\_1$ message first appears as a singleton in the first row where it was in the 0-block so its code word is just 0. No singletons appear in the second join (second row) so there are no two-letter code words in the developing code. Then in the third join (row 3) both $u\_4$ and $u\_5$ first appear as singletons in the Consecutive Joins column so their history of 0-blocks and 1-blocks (starting in row 1 Partitions column) give their codes of $110\; =\; u\_4$ and $111\; =\; u\_5$. Finally $u\_2$ and $u\_3$ appear in singletons in the final join (row 4) where all outcomes are singletons in $1\_U$, and their history of 0-blocks and 1-blocks gives their codes of $1000\; =\; u\_2$ and $1001\; =\; u\_3$. The history of each outcome or message to its singleton cannot be repeated for any other message (since singletons cannot further differentiate) so this procedure always generates what is called an instantaneous code where no code word can be the prefix of another code word [5] (pp. 62-64).

Figure 6 gives the ‘progress’ of an outcome starting with its undifferentiated form in the root of the ‘upside down’ tree (the indiscrete partition) and then traced out as each outcome or message code is implemented to finally yield the fully distinguished outcome, i.e., its singleton block in the Consecutive Joins column of Table 6.

The most famous code is, of course, the genetic code which is instantaneous so it can be generated by a sequence of partition joins. In this case, each partition has four blocks corresponding to the four code letters U, C, A, and G in the code alphabet. For the partitions in Figure 7, which correspond to the partitions in the Partitions column like in Table 6, the consecutive joins give all 64 singletons after three branchings or joins so the amino acids have 3-letter code words. Empirically, the code is redundant since there can be several codes for the same acid.

The circles in Figure 7 trace out the code for Thr4 (one of the code words for Thr, Threonine) which is ACG = Thr4. Note that the order of the partitions counts in the consecutive-joins determination of the genetic codes. A different ordering gives a different code which may not describe the operation of the DNA-RNA machinery to produce a certain amino acid from a given code word.

In terms of a tree diagram as in Figure 8, the tree would branch four ways at each branching point and there are three levels, so there are $4^3\; =\; 64$ leaves in the tree.

The generative mechanism associated with the genetic code is the whole DNA-RNA machinery that generates the amino acid as the output from the code word as the input. If we abstractly represent the DNA-RNA machinery as that tree with 64 leaves, then the given code word tells the machinery how to traverse the tree to arrive at the desired leaf.

Noam Chomsky's Principles & Parameters (P&P) mechanism ([43]; [44]) for language learning can be modeled as a generative mechanism. Again, we can consider a tree diagram where each branching point has a two-way switch to determine one grammatical rule or another in the language being acquired.

A simple image may help to convey how such a theory might work. Imagine that a grammar is selected (apart from the meanings of individual words) by setting a small number of switches - 20, say - either "On" or "Off." Linguistic information available to the child determines how these switches are to be set. In that case, a huge number of different grammars (here, 2 to the twentieth power) will be prelinguistically available, although a small amount of experience may suffice to fix one [45] (p. 154).

And the reference to 20 recalls the game of "20 questions" where the answers to the yes-or-no questions guides one closer and closer to the desired hidden answer. Chomsky uses the Higginbotham model to describe a Universal Grammar (UG) as a generative mechanism.

Many of these principles are associated with parameters that must be fixed by experience. The parameters must have the property that they can be fixed by quite simple evidence, because this is what is available to the child; the value of the head parameter, for example, can be determined from such sentences as John saw Bill (versus John Bill saw). Once the values of the parameters are set, the whole system is operative. Borrowing an image suggested by James Higginbotham, we may think of UG as an intricately structured system, but one that is only partially "wired up." The system is associated with a finite set of switches, each of which has a finite number of positions (perhaps two). Experience is required to set the switches. When they are set, the system functions [46] (p. 146).

In the tree modeling of the P&P approach, the relative poverty of linguistic experience that sets the switches plays the role of the code that guides the mechanism from the undifferentiated root state (all switches at neutral) to the final specific grammar represented as a leaf.

Most important of all, it offered an explanatory model for the empirical analyses which opened a way to meet the challenge of “Plato’s Problem” posed by children’s effortless “yet completely successful” acquisition of their grammars under the conditions of the poverty of the stimulus. This becomes particularly clear if we take the view that parametric variation exhausts the possible morphosyntactic variation among languages and further assume that there is a finite set of binary parameters. Imposing an arbitrary order on the parameters, a given language’s set of parameter settings can then be reduced to a series of 0s and 1s, i.e. a binary number $n$ [47] (p. 17).

The binary number $n$ is the code to traverse the tree down to the leaf representing the particular grammar.

The question about the acquisition of a grammar is a good topic to compare and contrast a selectionist mechanism with a generative mechanism. What would a selectionist approach to learning a grammar look like? A child would (perhaps randomly) generate a diverse range of babblings, some of which would be differentially reinforced or selected by the linguistic environment (e.g., [48]).

Skinner, for example, was very explicit about it. He pointed out, and he was right, that the logic of radical behaviorism was about the same as the logic of a pure form of selectionism that no serious biologist could pay attention to, but which is [a form of] popular biology – selection takes any path. And parts of it get put in behaviorist terms: the right paths get reinforced and extended, and so on. It’s like a sixth grade version of the theory of evolution. It can’t possibly be right. But he was correct in pointing out that the logic of behaviorism is like that [of naïve adaptationism], as did Quine [49] (Section 10).

A more sophisticated version of a selectionist model for the language-acquisition faculty or universal grammar (UG) could be called the format-selection (FS) approach (Chomsky, private communication). The diverse variants that are actualized in the mental mechanism are different sets of rules or grammars. Then given some linguistic input from the linguistic environment, the grammars are evaluated according to some evaluation metric, and the best rules are selected.

Universal grammar, in turn, contains a rule system that generates a set (or a search space) of grammars, $\backslash \{G\_1,\; G\_2,\; \backslash dots,\; G\_n\backslash \}$. These grammars can be constructed by the language learner as potential candidates for the grammar that needs to be learned. The learner cannot end up with a grammar that is not part of this search space. In this sense, UG contains the possibility to learn all human languages (and many more). ... The learner has a mechanism to evaluate input sentences and to choose one of the candidate grammars that are contained in his search space [50] (p. 292)

After a sufficient stream of linguistic inputs, the mechanism should converge to the best grammar that matches the linguistic environment. Since it is optimizing over sets of rules, this model at least takes seriously the need to account for the choice of rules (rather than just assuming the child can infer the rules from raw linguistic data). Early work (through the 1970s) on accounting for the language-acquisition faculty or universal grammar (UG) seems to have assumed such an approach. The problems that eventually arose with the FS approach could be seen as the conflict between descriptive and explanatory adequacy.

Since selection operates on actualities, in order to describe the enormous range of human language grammars, the range of grammars considered would make for an unfeasible computational load of evaluating the linguistic experience. If the range was restricted to make computation more feasible, then it would not explain the variety of human languages. Hence the claim is that the P&P generative mechanism gives a more plausible account of human language acquisition than a behavioral/selectionist approach.

Our simple partition lattice or rooted tree models of a generative mechanism pale beside the complexity of embryonic development. Nevertheless, it seems clear that the stem cells have the role of embodying the potentialities like the indiscrete partition $\backslash mathbf\{0\}\_U$ or the root in a rooted tree. Thus, the role of stem cells in the development of an embryo from a fertilized egg into a full organism can be modeled as a generative mechanism.

As illustrated in Figure 9, stem cells come in three general varieties: A) the stem cells that can reproduce undifferentiated copies of themselves, B) the stem cells that can reproduce but can also produce a somewhat differentiated cell, and C) a specialized differentiated cell. Each branching point in a tree has a certain number of possible leaves or terminal types of cells beneath it in the tree. In a division (#1) of an A-type cell, each of the resulting A-type cell could have a full set of leaves beneath it. But when it splits (#2) into another A-type cell and a B-type cell, then the B-cell has a restricted number of leaves beneath it. The B-type cells can split (#3) in two, and finally when a B-type cell gives rise (#4) to a specific C-type of cell, that is a terminal branch, i.e., a leaf, in the tree.

The codes that inform the progress through the tree are not fully understood, but apparently the positional epigenetic information in the developing embryo provides the information about the next development steps. In general terms,

[t]hat model harks back to the “developmental landscape” proposed by Conrad Waddington in 1956. He likened the process of a cell homing in on its fate to a ball rolling down a series of ever-steepening valleys and forked paths. Cells had to acquire more and more information to refine their positional knowledge over time — as if zeroing in on where and what they were through “the 20 questions game, according to Jané Kondev, a physicist at Brandeis University. [51]

Again, the reference to the game of 20 questions reveals the common generative mechanism of traversing a tree from the root to a specific leaf. Information is distinctions so more and more distinctions (“forked paths”) are made along a path like the path in the partition lattice from the one block in the indiscrete partition to smaller and smaller blocks until finally arriving at a singleton block in the discrete partition.

In Figure 10, the lattice of partitions on $U\; =\; \backslash \{a,\; b,\; c,\; d\backslash \}$ is represented using the shorthand of eliminating the innermost curly brackets in favor of juxtaposition so $\backslash \{\backslash \{a\backslash \},\; \backslash \{b,\; c,\; d\backslash \}\backslash \}$ is $\backslash \{a,\; bcd\backslash \}$. The path is indicated where the block containing the $b$ outcome is differentiated by more and more distinctions until finally becoming fully distinct as a singleton block in the discrete partition. The indicated path through the lattice of partitions is like the Consecutive Joins column in Tables 5 and 6. The increasing amount of information used to make all the differentiations is indicated by the rising logical entropies of the increasingly refined partitions.

Moreover, we have seen in the analysis of group representations that symmetries play the role of equivalences or indistinctions, and thus that the making of distinctions is described as “symmetry-breaking.” That holds true also in embryonic development.

Ultimately, symmetry breaking shapes your whole body, from the location of your head and toes to the position of your organs, from the symmetric location of lungs and kidneys to the way the heart is on the left. All this, in turn, derives from asymmetries on the molecular scale.

Symmetry breaking is essential to shape many of the most dramatic phases of our development [52] (p. 13).

Thus, it seems clear that the whole complex and only partly understood process of development from a stem cell to an fully differentiated organism can be described as a generative mechanism.

There is a long tradition in biological thought of juxtaposing selectionism, associated with Darwin, with instructionism, associated with Lamarck ([53]; [54]). In an instructionist or Lamarckian mechanism, the environment would transmit detailed instructions about a certain adaptation to an organism, while in a selectionist mechanism, a diverse variety of (random) variations would occur, and then some variations would be selected by the environment as the “survival of the fittest.” The discovery that the immune system was a selectionist mechanism [55] generated a wave of enthusiasm, a “Second Darwinian Revolution” [41], for selectionist theories [40].

In his Nobel Lecture [56], Niels Jerne even tried to draw parallels between Chomsky’s generative grammar and selectionism. One of the distinctive features of a selectionist mechanism is that the possibilities must be in some sense actualized or realized in order for selection to operate on and differentially amplify or select some of the actual variants while the others languish, atrophy, or die off. In the case of the human immune system, “It is estimated that even in the absence of antigen stimulation a human makes at least $10^\{15\}$ different antibody molecules—its preimmune antibody repertoire” [57] (p. 1221).

In Chomsky’s critique of a selectionist theory of universal grammar, he noted the computational infeasibility of having representations of all possible human grammars in order for linguistic experience and an evaluation criterion to perform a selective function on them. The analysis of Chomsky’s P&P theory as a generative mechanism instead suggests that the old juxtaposition of “selectionism versus instructionism” is not the most useful framing for the study of biological mechanisms. It is better framed as selectionist mechanisms versus generative mechanisms.

The discovery of the genetic code and DNA-RNA machinery for the production of amino acids powerfully showed the existence of another biological mechanism, a generative mechanism, that is quite distinct from a selectionist mechanism. The examples of Chomsky’s P&P theory of grammar acquisition and the role of stem cells in embryonic development provide more evidence of the importance of generative mechanisms.

To better illustrate these two main types of biological mechanisms, it might be useful to illustrate a selectionist and a generative mechanism in solving the same problem of determining one among the $8\; =\; 2^3$ options considered in Figure 10. The eight possible outcomes might be represented as: $|000\backslash rangle$, $|100\backslash rangle$, $|010\backslash rangle$, $|110\backslash rangle$, $|001\backslash rangle$, $|101\backslash rangle$, $|011\backslash rangle$, $|111\backslash rangle$.

In the selectionist scheme, all eight variants are in some sense actualized or realized in the initial state so that a fitness criterion or evaluation metric (as in the FS scheme) can operate on them. Some variants do better and some worse as indicated by the type size in Figure 11.

The “unfit” options dwindle, atrophy, or die off leaving the most fit option $|010\backslash rangle$ as the final outcome.

With the generative mechanism, the initial state (the root of the tree) is where all the switches are in neutral, so all the eight potential outcomes are in a “superposition” (between left and right) state indicated by the plus signs in the following Figure 12.

The initial experience or first letter in the code sets the first switch to the 0 option which reduces the state to $|000\backslash rangle\; +\; |001\backslash rangle\; +\; |010\backslash rangle\; +\; |011\backslash rangle$ (where the plus signs in the superposition of these options indicate that the second and third switches are still in neutral). Then subsequent experience sets the second switch to the 1 option and the third switch to the 0 option. Thus, we reach the same outcome $|010\backslash rangle$ as the final outcome in the two models but by quite different mechanisms. Note that the generative mechanism ‘selects’ or determines a specific outcome but that does not make it a ‘selectionist’ mechanism since it is making distinctions to turn an indefinite superposition-like state into a more definite state, as opposed to selecting between already existing variations according to a fitness criterion.

Another way to visually compare a selectionist mechanism with a generative mechanism is to consider a single-elimination (or knockout) tournament as a “red in tooth and claw” selectionist mechanism versus the implementation of a code for a specific leaf as a generative mechanism as in Figure 13. The selectionist mechanism starts with 8 existing teams and then binary contests whittle down the survivors to a eventual winner. The generative mechanism starts at the root, which like the superposition $\backslash mathbf\{0\}\_U$, embodies 8 possibilities and the sequence of binary-code switches will eventually distinguish the coded leaf. The fundamental (reverse-the-arrows) duality of category theory is turn-around-the-trees in this case of Figure 13.