Wisdom Of The Week

In the talk, Naftali Tishby, a computer scientist and neuroscientist from the Hebrew University of Jerusalem, presented evidence in support of a new theory explaining how deep learning works. Tishby argues that deep neural networks learn according to a procedure called the “information bottleneck,” which he and two collaborators first described in purely theoretical terms in 1999. The idea is that a network rids noisy input data of extraneous details as if by squeezing the information through a bottleneck, retaining only the features most relevant to general concepts. Striking new computer experiments by Tishby and his student Ravid Shwartz-Ziv reveal how this squeezing procedure happens during deep learning, at least in the cases they studied.

Tishby’s findings have the AI community buzzing. “I believe that the information bottleneck idea could be very important in future deep neural network research,” said Alex Alemi of Google Research, who has already developed new approximation methods for applying an information bottleneck analysis to large deep neural networks. The bottleneck could serve “not only as a theoretical tool for understanding why our neural networks work as well as they do currently, but also as a tool for constructing new objectives and architectures of networks,” Alemi said.

- New Theory Cracks Open the Black Box of Deep Learning

Quote of the Day

Reading furnishes the mind only with materials of knowledge; it is thinking that makes what we read ours.

- John Locke

Quote of the Day

Here's to the crazy ones. The misfits. The rebels. The troublemakers. The round pegs in the square holes. The ones who see things differently. They're not fond of rules. And they have no respect for the status quo. You can quote them, disagree with them, glorify or vilify them. About the only thing you can't do is ignore them. Because they change things. They push the human race forward. And while some may see them as the crazy ones, we see genius. Because the people who are crazy enough to think they can change the world, are the ones who do.

- Rob Siltanen

Quote of the Day

That as people age, accumulate more and more private experiences, their sense of history tightens, narrows, becomes more personal? So that to the extent that they remember events of social importance, they remember only for example 'where they were' when such-and-such occurred. Et cetera et cetera. Objective events and data become naturally more and more subjectively colored.

- David Foster Wallace, The Broom of the System

Quote of the Day

Make your ego porous. Will is of little importance, complaining is nothing, fame is nothing. Openness, patience, receptivity, solitude is everything.

- Rainer Maria Rilke

Quote of the Day

We are trying to prove ourselves wrong as quickly as possible, because only in that way can we find progress.

- Richard Feynman

Quote of the Day

A very disturbing feature of overconfidence is that it often appears to be poorly associated with knowledge - that is, the more ignorant the individual, the more confident he or she might be.

- Robert Trivers, Deceit and Self-Deception: Fooling Yourself the Better to Fool Others

Quote of the Day

Wisdom cannot be imparted. Wisdom that a wise man attempts to impart always sounds like foolishness to someone else ... Knowledge can be communicated, but not wisdom. One can find it, live it, do wonders through it, but one cannot communicate and teach it.

- Hermann Hesse, Siddhartha

Wisdom Of The Week

When the butterfly emerged from its pupa, Robert Reed was stunned. It was a Gulf fritillary—a bright-orange species with a few tigerlike stripes. But this butterfly had no trace of orange anywhere. It was entirely black and silver. “It was the most heavy-metal butterfly I’ve ever seen,” Reed says. “It was amazing to see that thing crawl out of the pupa.”

Reed’s team at Cornell University had created the metal butterfly by deleting just one of its genes, using the revolutionary gene-editing technique known as CRISPR. And by performing the same feat across several butterfly species, the team showed that this one gene, known as optix, controls all kinds of butterfly patterns. Red becomes black. Matte becomes shiny. Another gene, known as WntA, produces even wilder variations when it’s deleted. Eyespots disappear. Boundaries shift. Stripes blur.

These experiments prove what earlier studies had suggested—that optix and WntA are “paintbrush genes,” says Anyi Mazo-Vargas, one of Reed’s students. “Wherever you put them, you’ll have a pattern.”

[---]

The details are still unclear though. How do these gene networks get rewired? When did they take on their roles as master regulators of wing patterns? Why have only a few genes done so? Did butterflies recruit these genes to paint their wings once during their evolution, or many times independently? A decade ago, these would have been fanciful questions. But in the CRISPR era, it suddenly seems possible to answer them.

“CRISPR is a miracle,” Reed says. “The first time we tried it, it worked, and when I saw that butterfly come out ... the biggest challenge of my career had just turned into an undergraduate project.”

- Scientists Can Now Repaint Butterfly Wings

Quote of the Day