Ambiguously Slobbering Dogs

I was initially dismissive of this note from Google Research on improving machine translation via Deep Learning Networks by adding in a sentence-level network. My goodness, they’ve rediscovered anaphora and co-reference resolution! Next thing they will try is some kind of network-based slot-filler ontology to carry gender metadata. But their goal was to add a framework to their existing recurrent neural network architecture that would support a weak, sentence-level resolution of translational ambiguities while still allowing the TPU/GPU accelerators they have created to function efficiently. It’s a hack, but one that potentially solves yet another corner of the translation problem and might result in a few percent further improvements in the quality of the translation.

But consider the following sentences:

The dog had the ball. It was covered with slobber.

The dog had the ball. It was thinking about lunch while it played.

In these cases, the anaphora gets resolved by semantics and the resolution seems largely an automatic and subconscious process to us as native speakers. If we had to translate these into a second language, however, we would be able to articulate that there are specific reasons for correctly assigning the “It” to the ball in the first two sentences. Well, it might be possible for the dog to be covered with slobber, but we would guess the sentence writer would intentionally avoid that ambiguity. The second set of sentences could conceivably be ambiguous if, in the broader context, the ball was some intelligent entity controlling the dog. Still, when our guesses are limited to the sentence pairs in isolation we would assign the obvious interpretations. Moreover, we can resolve giant, honking passage-level ambiguities with ease, where the author is showing off in not resolving the co-referents until obscenely late in the text.

In combination, we can see the obvious problem with sentence-level “attention” calculations. The context has to be moving and fairly long.

Gravity and the Dark Star

Totality in Nebraska

I began at 5 AM from the Broomfield Aloft hotel, strategically situated in a sterile “new urban” office park cum apartment complex along the connecting freeway between Denver and Boulder. The whole weekend was fucked in a way: colleges across Colorado were moving in for a Monday start, half of Texas was here already, and most of Colorado planned to head north to the zone of totality. I split off I-25 around Loveland and had success using US 85 northbound through Cheyenne. Continuing up 85 was the original plan, but that fell apart when 85 came to a crawl in the vast prairie lands of Wyoming. I dodged south and east, then, (dodging will be a continuing theme) and entered Nebraska’s panhandle with middling traffic.

I achieved totality on schedule north of Scottsbluff. And it was spectacular. A few fellow adventurers were hanging out along the outflow lane of an RV dump at a state recreation area. One guy flew his drone around a bit. Maybe he wanted B roll for other purposes. I got out fast, but not fast enough, and dodged my way through lane closures designed to provide access from feeder roads. The Nebraska troopers were great, I should add, always willing to wave to us science and spectacle immigrants. Meanwhile, SiriusXM spewed various Sibelius pieces that had “sun” in their name, while the Grateful Dead channel gave us a half dozen versions of Dark Star, the quintessential jam song for the band that dates to the early, psychedelic era of the band.

Was it worth it? I think so, though one failed dodge that left me in a ten mile bumper-to-bumper crawl in rural Nebraska with a full bladder tested my faith in the stellar predictability of gravity. Gravity remains an enigma in many ways, though the perfection of watching the corona flare around the black hole sun shows just how unenigmatic it can be in the macroscopic sphere.

But reconciling gravity with quantum-scale phenomena remains remarkably elusive and is the beginning of the decades-long detour through string theory which, admittedly, some have characterized as “fake science” due to our inability to find testable aspects of the theory. Yet, there are some interesting recent developments that, though they are not directly string theoretic, have a relationship to the quantum symmetries that, in turn, led to stringiness.

So I give you Juan Maldacena and Leonard Susskind’s suggestion that ER = ERP. This is a rather remarkable conclusion that unites quantum and relativistic realities, but is based on a careful look at the symmetry between two theoretical outcomes at the two different scales. So how does it work? In a nut shell, the claim is that quantum entanglement is identical to relativistic entanglement. Just like the science fiction idea of wormholes connecting distant things together to facilitate faster-than-light travel, ER connects singularities like black holes together. And the correlations that occur between black holes is just like the correlations between entangled quanta. Neither is amenable to either FTL travel or signaling due to Lorentzian traversability issues (former) or Bell’s Inequality (latter).

Today was just a shadow, classically projected, maybe just slightly twisted by the gravity wells, not some wormhole wending its way through space and time. It is worth remembering, though, that the greatest realization of 20th century physics is that reality really isn’t in accord with our everyday experiences. Suns and moons kind of are, briefly and ignoring fusion in the sun, but reality is almost mystically entangled with itself, a collection of vibrating potentialities that extend out everywhere, and then, unexpectedly, the potentialities are connected together in another way that defies these standard hypothetical representations and the very notion of space connectivities.

Fantastical Places and the Ethics of Architecture

Lemuria was a hypothetical answer to the problem of lemurs in Madagascar and India. It was a connective tissue for the naturalism observed during the formative years of naturalism itself. Only a few years had passed since Darwin’s Origin of the Species came out and the patterns of observations that drove Darwin’s daring hypothesis were resonating throughout the European intellectual landscape. Years later, the Pangaea supercontinent would replace the temporary placeholder of Lemuria and the concept would be relegated to mythologized abstractions alongside Atlantis and, well, Hyperborea.

I’m in Lemuria right now, but it is a different fantastical place. In this case, I’m in the Lemuria Earthship Biotecture near Taos, New Mexico. I rented it out on a whim. I needed to travel to Colorado to drop off some birthday cards for our son and thought I might come by and observe this ongoing architectural experiment that I’ve been tracking for decades but never visited. I was surprised to find that I could rent a unit.

First, though, you have to get here, which involves crossing the Rio Grande Gorge:

Once I arrived, I encountered throngs of tourists, including an extended Finnish family that I had to eavesdrop on to guess the language they were speaking. The Earthship project has a long history, but it is always a history of trying to create sustainable, off-the-grid structures that maximize the use of disposable aspects of our society. So the walls are tires filled with dirt or cut wine bottles embedded in cement. Photovoltaics charge batteries and gray water (shower and washing water) is reused to flush toilets and grow food plants. Black water (toilet water) flows into leachfields that support landscape plants. Rainwater is captured from the roof to fill the gray water reservoirs. And, amazingly, it all works very well.

Here’s my video on arrival at Lemuria. There is wind noise when I’m on the roof, but it dies off when I get inside.

Architecture and ethics have always had an uneasy truce. At the most basic, there are the ethical limits of not deceiving clients about materials, costs, or functionality. But the harder questions build around aesthetic value versus functional value. A space that is sculptural like a Calatrava train station or Frank Gehry music hall is a space that values aesthetics at least as highly as functionality. There is no reusability in curved magnesium panels.

Where experiements like the Earthship thrive is in finding a weighted balance that gives functional and sustainable solutions precedence over the purely conceptual aspects of architecture. What could be is grounded by ethical stewardship.

Lemuria is standing up to a heavy downpour quite well right now as the monsoonal storms lash over the high plateau. I think I can hear the water flowing into the cisterns and an occasional pump pushing water through filters. It almost seems more fantastical that we don’t build houses like this.

Bright Sarcasm in the Classroom

That old American tradition, the Roman Salute

When a Pew research poll discovered a shocking divide between self-identifying Republicans/GOP-leaning Independents and their Democratic Party opposites on the question of the value of higher education, the commentariat went apeshit. Here’s a brief rundown of sources, left, center, and right, and what they decided are the key issues:

  • National Review: Higher education has eroded the Western canon and turned into a devious plot to rob our children of good thinking, spiked with avocado toast.
  • Paul Krugman at New York Times: Conservative tribal identification leads to opposition to climate change science or evolution, and further towards a “grim” anti-intellectualism.
  • New Republic: There is no evidence that college kid’s political views are changed by higher education and, also, that conservative-minded professors aren’t much maltreated on campus either, so the conservative complaints are just overblown anti-liberal hype that, they point out, has some very negative consequences.

I would make a slightly more radical claim than Krugman, for instance, and one that is pointedly opposed to Simonson at National Review. In higher education we see not just a dedication to science but an active program of criticizing and deconstructing ideas like the Western canon as central to higher thought. In history, great man theories have been broken down into smart and salient compartments that explore the many ways in which groups and individuals, genders and ideas, all were part of fashioning the present. These changes, largely late 20th century academic inventions, have broken up the monopolies on how concepts of law, order, governance, and the worth of people were once formulated. This must be anti-conservative in the pure sense that there is little to be conserved from older ideas, except as objects of critique. We need only stroll through the grotesque history of Social Darwinism, psychological definitions of homosexuality as a mental disorder, or anthropological theories of race and values to get a sense for why academic pursuits, in becoming more critically influenced by a burgeoning and democratizing populace, were obligated to refine what is useful, intellectually valuable, and less wrong. The process will continue, too.

The consequences are far reaching. Higher education correlates necessarily with liberal values and those values tend to correlate more with valuing reason and fairness over tradition and security. That means that atheism has a greater foothold and science as a primary means of truth discovery takes precedence over the older and uglier angels of our nature. The enhanced creativity that arises from better knowledge of the world and accurate and careful assessment then, in turn, leads to knowledge generation and technological innovation that is derived almost exclusively from a broad engagement with ideas. This can cause problems when ordering Italian sandwiches.

Is there or should there be any antidote to the disjunctive opinions on the value of higher learning? Polarized disagreements on the topic can lead to societal consequences that are reactive and precipitous, which is what all three sources are warning about in various ways. But the larger goals of conservatives should be easily met through the mechanism that most of them would agree is always open: form, build, and attend ideologically-attuned colleges. There are at least dozens of Christian colleges that have various charters that should meet some of their expectations. If these institutions are good for them and society as a whole, they just need to do a better job of explaining that to America. Then, like the consumer flocking from Microsoft to Apple, the great public and private institutions will lose the student debt dollar to these other options and, finally, indoctrination in all that bright sarcasm will end in the classroom. Maybe, then, everyone will agree that the earth is only a few thousand years old and that coal demand proceeds from supply.

Less Dead

I’m feeling less dead than I could be. Here’s the rattlesnake that struck and bounced off my running shoe this morning:

He started rattling after the initial strike, which seems like an evolutionary spandrel. At least he didn’t have a machine gun. I’ve named him Bartholomew and wish him the best on his future journeys. And here is the juvenile oryx who was laughing at the situation nearby:

Zebras with Machine Guns

I was just rereading some of the literature on Plantinga’s Evolutionary Argument Against Naturalism (EAAN) as a distraction from trying to write too much on ¡Reconquista!, since it looks like I am on a much faster trajectory to finishing the book than I had thought. EAAN is a curious little argument that some have dismissed as a resurgent example of scholastic theology. It has some newer trappings that we see in modern historical method, however, especially in the use Bayes’ Theorem to establish the warrant of beliefs by trying to cast those warrants as probabilities.

A critical part of Plantinga’s argument hinges on the notion that evolutionary processes optimize against behavior and not necessarily belief. Therefore, it is plausible that an individual could hold false beliefs that are nonetheless adaptive. For instance, Plantinga gives the example of a man who desires to be eaten by tigers but always feels hopeless when confronted by a given tiger because he doesn’t feel worthy of that particular tiger, so he runs away and looks for another one. This may seem like a strange conjunction of beliefs and actions that happen to result in the man surviving, but we know from modern psychology that people can form elaborate justifications for perceived events and wild metaphysics to coordinate those justifications.

If that is the case, for Plantinga, the evolutionary consequence is that we should not trust our belief in our reasoning faculties because they are effectively arbitrary. There are dozens of responses to this argument that dissect it from many different dimensions. I’ve previously showcased Branden Fitelson and Elliot Sober’s Plantinga’s Probability Arguments Against Evolutionary Naturalism from 1997, which I think is one of the most complete examinations of the structure of the argument. There are two critical points that I think emerge from Fitelson and Sober. First, there is the sober reminder of the inherent frailty of scientific method that needs to be kept in mind. Science is an evolving work involving many minds operating, when at its best, in a social network that reduces biases and methodological overshoots. It should be seen as a tentative foothold against “global skepticism.”

The second, and critical take-away from that response is more nuanced, however. The notion that our beliefs can be arbitrarily disconnected from adaptive behavior in an evolutionary setting, like the tiger survivor, requires a very different kind of evolution than we theorize. Fitelson and Sober point out that if anything was possible, zebras might have developed machine guns to defend against lions rather than just cryptic stripes. Instead, the sieve of possible solutions to adaptive problems is built on the genetic and phenotypic variants that came before. This will limit the range of arbitrary, non-true beliefs that can be compatible with an adaptive solution. If the joint probability of true belief and adaptive behavior is much higher than the alternative, which we might guess is true, then there is a greater probability that our faculties are reliable. In fact, we could argue that using a parsimony argument that extends Bayesian analysis to the general case of optimal inductive models (Sober actually works on this issue extensively), that there are classes of inductive solutions that, through eliminating add-ons, outperform predictively those solutions that have extra assumptions and entities. So, P(not getting eaten | true belief that tigers are threats) >> P(not getting eaten | false beliefs about tigers), especially when updated over time. I would be remiss if I didn’t mention that William of Ockham of Ockham’s Razor-fame was a scholastic theologian, so if Plantinga’s argument is revisiting those old angels-head-pin-style arguments, it might be opposed by a fellow scholastic.

¡Reconquista! at 50K

¡Reconquista! has taken on that magical quality of momentum where it is almost writing itself. Or maybe it’s just that satire, bleak and horrifying, is the perfect mood for the times. These counts do not include early plot development and notes, which read out at another 4K or more, depending on how you factor it.

The analytics put me on an exit trajectory around mid-August.

Brain Gibberish with a Convincing Heart

Elon Musk believes that direct brain interfaces will help people better transmit ideas to one another in addition to just allowing thought-to-text generation. But there is a fundamental problem with this idea. Let’s take Hubert Dreyfus’ conception of the way meaning works as being tied to a more holistic view of our social interactions with others. Hilary Putnam would probably agree with this perspective, though now I am speaking for two dead philosphers of mind. We can certainly conclude that my mental states when thinking about the statement “snow is white” are, borrowing from Putnam who borrows from Quine, different from a German person thinking “Schnee ist weiß.” The orthography, grammar, and pronunciation are different to begin with. Then there is what seems to transpire when I think about that statement: mild visualizations of white snow-laden rocks above a small stream for instance, or, just now, Joni Mitchell’s “As snow gathers like bolts of lace/Waltzing on a ballroom girl.” The centrality or some kind of logical ground that merely asserts that such a statement is a propositional truth that is shared in some kind of mind interlingua doesn’t bear much fruit to the complexities of what such a statement entails.

Religious and political terminology is notoriously elastic. Indeed, for the former, it hardly even seems coherent to talk about the concept of supernatural things or events. If they are detectable by any other sense than some kind of unverifiable gnosis, then they are at least natural in that they are manifesting in the observable world. So supernatural imposes a barrier that seems to preclude any kind of discussion using ordinary language. The only thing left is a collection of metaphysical assumptions that, in lacking any sort of reference, must merely conform to the patterns of synonymy, metonymy, and other language games that we ordinarily reserve for discernible events and things. And, of course, where unverifiable gnosis holds sway, it is not public knowledge and therefore seems to mainly serve as a social mechanism for attracting attention to oneself.

Politics takes on a similar quality, with it often said to be a virtue if a leader can translate complex policies into simple sound bites. But, as we see in modern American politics, what instead happens is that abstract fear signaling is the primary currency to try to motivate (and manipulate) the voter. The elasticity of a concept like “freedom” is used to polarize the sides of political negotiation that almost always involves the management of winners and losers and the dividing line between them. Fear mixes with complex nostalgia about times that never were, or were more nuanced than most recall, and jeremiads serve to poison the well of discourse.

So, if I were to have a brain interface, it might be trainable to write words for me by listening to the regular neural firing patterns that accompany my typing or speaking, but I doubt it would provide some kind of direct transmission or telepathy between people that would have any more content than those written or spoken forms. Instead, the inscrutable and non-referential abstractions about complex ideas would be tied together and be in contrast with the existing holistic meaning network. And that would just be gibberish to any other mind. Worst still, such a system might also be able to convey raw emotion from person to person, thus just amplifying the fear or joy component of the idea without being able to transmit the specifics of the thoughts. And that would be worse than mere gibberish, it would be gibberish with a convincing heart.

Inclement Science

Found at 6,500 feet in New Mexico’s Organ Mountains this morning, driven into an old log, facing White Sands Missile Range:

Can’t help but think it is a statement on the threat to climate science and missions like Jason-3, but someone likely just lost it on the trail and a good soul pushed the pin into the wood for potential rediscovery.

The Obsessive Dreyfus-Hawking Conundrum

I’ve been obsessed lately. I was up at 5 A.M. yesterday and drove to Ruidoso to do some hiking (trails T93 to T92, if interested). The San Augustin Pass was desolate as the sun began breaking over, so I inched up into triple digit speeds in the M6. Because that is what the machine is made for. Booming across White Sands Missile Range, I recalled watching base police work with National Park Rangers to chase oryx down the highway while early F117s practiced touch-and-gos at Holloman in the background, and then driving my carpool truck out to the high energy laser site or desert ship to deliver documents.

I settled into Starbucks an hour and a half later and started writing on ¡Reconquista!, cranking out thousands of words before trying to track down the trailhead and starting on my hike. (I would have run the thing but wanted to go to lunch later and didn’t have access to a shower. Neither restaurant nor diners deserve an après-run moi.) And then I was on the trail and I kept stopping and taking plot and dialogue notes, revisiting little vignettes and annotating enhancements that I would later salt in to the main text over lunch. And I kept rummaging through the development of characters, refining and sifting the facts of their lives through different sets of sieves until they took on both a greater valence within the story arc and, often, more comedic value.

I was obsessed and remain so. It is a joyous thing to be in this state, comparable only to working on large-scale software systems when the hours melt away and meals slip as one cranks through problem after problem, building and modulating the subsystems until the units begin to sing together like a chorus. In English, the syntax and semantics are less constrained and the pragmatics more pronounced, but the emotional high is much the same.

With the recent death of Hubert Dreyfus at Berkeley it seems an opportune time to consider the uniquely human capabilities that are involved in each of these creative ventures. Uniquely, I suggest, because we can’t yet imagine what it would be like for a machine to do the same kinds of intelligent tasks. Yet, from Stephen Hawking through to Elon Musk, influential minds are worried about what might happen if we develop machines that rise to the level of human consciousness. This might be considered a science fiction-like speculation since we have little basis for conjecture beyond the works of pure imagination. We know that mechanization displaces workers, for instance, and think it will continue, but what about conscious machines?

For Dreyfus, the human mind is too embodied and situational to be considered an encodable thing representable by rules and algorithms. Much like the trajectory of a species through an evolutionary landscape, the mind is, in some sense, an encoded reflection of the world in which it lives. Taken further, the evolutionary parallel becomes even more relevant in that it is embodied in a sensory and physical identity, a product of a social universe, and an outgrowth of some evolutionary ping pong through contingencies that led to greater intelligence and self-awareness.

Obsession with whatever cultivars, whatever traits and tendencies, lead to this riot of wordplay and software refinement is a fine example of how this moves away from the fears of Hawking and towards the impossibilities of Dreyfus. We might imagine that we can simulate our way to the kernel of instinct and emotion that makes such things possible. We might also claim that we can disconnect the product of the effort from these internal states and the qualia that defy easy description. The books and the new technologies have only desultory correspondence to the process by which they are created. But I doubt it. It’s more likely that getting from great automatic speech recognition or image classification to the general AI that makes us fearful is a longer hike than we currently imagine.