Active Deep Learning

BrainDeep Learning methods that use auto-associative neural networks to pre-train (with bottlenecking methods to ensure generalization) have recently been shown to perform as well and even better than human beings at certain tasks like image categorization. But what is missing from the proposed methods? There seem to be a range of challenges that revolve around temporal novelty and sequential activation/classification problems like those that occur in natural language understanding. The most recent achievements are more oriented around relatively static data presentations.

Jürgen Schmidhuber revisits the history of connectionist research (dating to the 1800s!) in his October 2014 technical report, Deep Learning in Neural Networks: An Overview. This is one comprehensive effort at documenting the history of this reinvigorated area of AI research. What is old is new again, enhanced by achievements in computing that allow for larger and larger scale simulation.

The conclusions section has an interesting suggestion: what is missing so far is the sensorimotor activity loop that allows for the active interrogation of the data source. Human vision roams over images while DL systems ingest the entire scene. And the real neural systems have energy constraints that lead to suppression of neural function away from the active neural clusters.

Read the rest

The Great Crustacean

little-lobster-costumeDavid Foster Wallace’s Joseph Frank’s Dostoevsky in Consider the Lobster is worth reading for nothing else than the following two paragraphs:

The big thing that makes Dostoevsky invaluable for American readers and writers is that he appears to possess degrees of passion, conviction, and engagement with deep moral issues that we—here, today—cannot or do not permit ourselves. Joseph Frank does an admirable job of tracing out the interplay of factors that made this engagement possible—[Dostoevsky]’s own beliefs and talents, the ideological and aesthetic climates of his day, etc. Upon his finishing Frank’s books, though, I think that any serious American reader/writer will find himself driven to think hard about what exactly it is that makes many of the novelists of our own place and time look so thematically shallow and lightweight, so morally impoverished, in comparison to Gogol or Dostoevsky (or even to lesser lights like Lermontov and Turgenev). Frank’s bio prompts us to ask ourselves why we seem to require of our art an ironic distance from deep convictions or desperate questions, so that contemporary writers have to either make jokes of them or else try to work them in under cover of some formal trick like intertextual quotation or incongruous juxtaposition, sticking the really urgent stuff inside asterisks as part of some multivalent defamiliarization-flourish or some such shit.

Part of the explanation for our own lit’s thematic poverty obviously includes our century and situation. The good old modernists, among their other accomplishments, elevated aesthetics to the level of ethics—maybe even metaphysics—and Serious Novels after Joyce tend to be valued and studied mainly for their formal ingenuity. Such is the modernist legacy that we now presume as a matter of course that “serious” literature will be aesthetically distanced from real lived life.

Read the rest