Wisdom Of The Week

In the talk, Naftali Tishby, a computer scientist and neuroscientist from the Hebrew University of Jerusalem, presented evidence in support of a new theory explaining how deep learning works. Tishby argues that deep neural networks learn according to a procedure called the “information bottleneck,” which he and two collaborators first described in purely theoretical terms in 1999. The idea is that a network rids noisy input data of extraneous details as if by squeezing the information through a bottleneck, retaining only the features most relevant to general concepts. Striking new computer experiments by Tishby and his student Ravid Shwartz-Ziv reveal how this squeezing procedure happens during deep learning, at least in the cases they studied.

Tishby’s findings have the AI community buzzing. “I believe that the information bottleneck idea could be very important in future deep neural network research,” said Alex Alemi of Google Research, who has already developed new approximation methods for applying an information bottleneck analysis to large deep neural networks. The bottleneck could serve “not only as a theoretical tool for understanding why our neural networks work as well as they do currently, but also as a tool for constructing new objectives and architectures of networks,” Alemi said.

- New Theory Cracks Open the Black Box of Deep Learning

Quote of the Day

Reading furnishes the mind only with materials of knowledge; it is thinking that makes what we read ours.

- John Locke

Quote of the Day

Here's to the crazy ones. The misfits. The rebels. The troublemakers. The round pegs in the square holes. The ones who see things differently. They're not fond of rules. And they have no respect for the status quo. You can quote them, disagree with them, glorify or vilify them. About the only thing you can't do is ignore them. Because they change things. They push the human race forward. And while some may see them as the crazy ones, we see genius. Because the people who are crazy enough to think they can change the world, are the ones who do.

- Rob Siltanen

Quote of the Day

That as people age, accumulate more and more private experiences, their sense of history tightens, narrows, becomes more personal? So that to the extent that they remember events of social importance, they remember only for example 'where they were' when such-and-such occurred. Et cetera et cetera. Objective events and data become naturally more and more subjectively colored.

- David Foster Wallace, The Broom of the System

Quote of the Day

Make your ego porous. Will is of little importance, complaining is nothing, fame is nothing. Openness, patience, receptivity, solitude is everything.

- Rainer Maria Rilke

Quote of the Day

We are trying to prove ourselves wrong as quickly as possible, because only in that way can we find progress.

- Richard Feynman

Quote of the Day

A very disturbing feature of overconfidence is that it often appears to be poorly associated with knowledge - that is, the more ignorant the individual, the more confident he or she might be.

- Robert Trivers, Deceit and Self-Deception: Fooling Yourself the Better to Fool Others

Quote of the Day

Wisdom cannot be imparted. Wisdom that a wise man attempts to impart always sounds like foolishness to someone else ... Knowledge can be communicated, but not wisdom. One can find it, live it, do wonders through it, but one cannot communicate and teach it.

- Hermann Hesse, Siddhartha

Wisdom Of The Week

When the butterfly emerged from its pupa, Robert Reed was stunned. It was a Gulf fritillary—a bright-orange species with a few tigerlike stripes. But this butterfly had no trace of orange anywhere. It was entirely black and silver. “It was the most heavy-metal butterfly I’ve ever seen,” Reed says. “It was amazing to see that thing crawl out of the pupa.”

Reed’s team at Cornell University had created the metal butterfly by deleting just one of its genes, using the revolutionary gene-editing technique known as CRISPR. And by performing the same feat across several butterfly species, the team showed that this one gene, known as optix, controls all kinds of butterfly patterns. Red becomes black. Matte becomes shiny. Another gene, known as WntA, produces even wilder variations when it’s deleted. Eyespots disappear. Boundaries shift. Stripes blur.

These experiments prove what earlier studies had suggested—that optix and WntA are “paintbrush genes,” says Anyi Mazo-Vargas, one of Reed’s students. “Wherever you put them, you’ll have a pattern.”

[---]

The details are still unclear though. How do these gene networks get rewired? When did they take on their roles as master regulators of wing patterns? Why have only a few genes done so? Did butterflies recruit these genes to paint their wings once during their evolution, or many times independently? A decade ago, these would have been fanciful questions. But in the CRISPR era, it suddenly seems possible to answer them.

“CRISPR is a miracle,” Reed says. “The first time we tried it, it worked, and when I saw that butterfly come out ... the biggest challenge of my career had just turned into an undergraduate project.”

- Scientists Can Now Repaint Butterfly Wings

Quote of the Day

Quote of the Day

I'm a greater believer in luck, and I find the harder I work the more I have of it.

- Thomas Jefferson

Quote of the Day

Quote of the Day

Quote of the Day

Quote of the Day

Quote of the Day

The planting of a tree, especially one of the long-living hardwood trees, is a gift which you can make to posterity at almost no cost and with almost no trouble, and if the tree takes root it will far outlive the visible effect of any of your other actions, good or evil.

- George Orwell

Wisdom Of The Week

At the center of the experiment was the plant Mimosa pudica, which has a dramatic response to unfamiliar mechanical stimuli: Its leaves fold closed, perhaps to scare away eager herbivores. Using a specially designed rail, Gagliano introduced her M. pudica to a new experience. She dropped them, as if they were on a thrill ride in an amusement park for plants. The mimosa plants reacted. Their leaves shut tight. But as Gagliano repeated the stimulus—seven sets of 60 drops each, all in one day—the plants’ response changed. Soon, when they were dropped, they didn’t react at all. It wasn’t that they were worn out: When she shook them, they still shut their leaves tight. It was as if they knew that being dropped was nothing to freak out about.

Three days later, Gagliano came back to the lab and tested the same plants again. Down they went, and … nothing. The plants were just as stoic as before.

This was a surprise. In studies of animals such as bees, a memory that sticks for 24 hours is considered long-term. Gagliano wasn’t expecting the plants to keep hold of the training days later. “Then I went back six days later, and did it again, thinking surely now they forgot,” she says. “Instead, they remembered, exactly as if they had just received the training.”

She waited a month and dropped them again. Their leaves stayed open. According to the rules that scientists routinely apply to animals, the mimosa plants had demonstrated that they could learn.
In the study of the plant kingdom, a slow revolution is underway. Scientists are beginning to understand that plants have abilities, previously unnoticed and unimagined, that we’ve only ever associated with animals. In their own ways, plants can see, smell, feel, hear, and know where they are in the world. One recent study found that clusters of cells in plant embryos act a lot like brain cells and help the embryo to decide when to start growing.

[---]

In their experiments, however, Crisp and Eichten don’t observe many plant memories being formed. What if, they ask, plant memory is rare simply because it’s better for plants to forget? “Having a memory, keeping track molecularly of signals that you’ve received in the past from your environment, does have a cost,” says Eichten. “Since we don’t see memories all that often … maybe plants don’t want to remember things all the time. Maybe it’s better to put their energies elsewhere.” Even when memories do form, they can fade. Another research group has shown, for example, that a plant might form an epigenetic memory of salt stress and pass it along across generations, but that if the stress fades, so does the memory. A plant that remembers too much might sacrifice healthy growth to be constantly on guard against drought, flood, salt, insects. Better, perhaps, to let those negative experiences go, instead of always preparing for the worst.

[---]

“In that context, memory is actually not the interesting bit—of course you have memory, otherwise you wouldn’t be able to do the trick,” she says. “Memory is part of the learning process. But—who is doing the learning? What is actually happening? Who is it that is actually making the association between fan and light?”

It’s telling that Gagliano uses the word “who,” which many people would be unlikely to apply to plants. Even though they’re alive, we tend to think of plants as objects rather than dynamic, breathing, growing beings. We see them as mechanistic things that react to simple stimuli. But to some extent, that’s true of every type of life on Earth. Everything that lives is a bundle of chemicals and electrical signals in dialogue with the environment in which it exists. A memory, such as of the heat of summer on last year’s beach vacation, is a biochemical marker registered from a set of external inputs. A plant’s epigenetic memory, of the cold of winter months, on a fundamental level, is not so different.- The Hidden Memories of Plants

Quote of the Day

To love. To be loved. To never forget your own insignificance. To never get used to the unspeakable violence and the vulgar disparity of life around you. To seek joy in the saddest places. To pursue beauty to its lair. To never simplify what is complicated or complicate what is simple. To respect strength, never power. Above all, to watch. To try and understand. To never look away. And never, never to forget.

- Arundhati Roy, The Cost of Living

Quote of the Day

Inspiration is for amateurs — the rest of us just show up and get to work. And the belief that things will grow out of the activity itself and that you will, through work, bump into other possibilities and kick open other doors that you would never have dreamt of if you were just sitting around looking for a great ‘art idea’.

- Chuck Close

Quote of the Day

Finish each day and be done with it. You have done what you could. Some blunders and absurdities no doubt crept in; forget them as soon as you can. Tomorrow is a new day. You shall begin it serenely and with too high a spirit to be encumbered with your old nonsense.

- Ralph Waldo Emerson

Quote of the Day

The excluded when on living on the fringe, like lepers, of whom true leper are only the illustration ordained by god to make us understand this wondrous parable, so that in saying “lepers” we would understand “outcast, poor, simple, excluded, uprooted from the countryside, humiliated in the cities” but we did not understand; the mystery of leprosy has continued to haunt us because we have not recognized the nature of the sign. Excluded as they were from the flock, all of them were ready to hear, or to produce, every sermon that, harking back to the words of Christ, would condemn the behaviour of the dogs and shepherds and would promise their punishment one day. The powerful have always realised this. The recovery of the outcasts demanded a reduction of the privileges of the powerful, so the excluded who became aware of their exclusion had to be branded as heretics, whatever their doctrine. This is the illusion of heresy. Everyone is heretical, everyone is orthodox. The faith a movement proclaims doesn’t count: what counts is the hope it offers.

- Umberto Eco

Quote of the Day

When it came time for me to give my talk on the subject, I started off by drawing an outline of the cat and began to name the various muscles.

The other students in the class interrupt me: "We *know* all that!"

"Oh," I say, "you *do*? Then no *wonder* I can catch up with you so fast after you've had four years of biology." They had wasted all their time memorizing stuff like that, when it could be looked up in fifteen minutes.”

- Richard Feynman, Surely You're Joking, Mr. Feynman!: Adventures of a Curious Character

Quote of the Day

The single greatest lesson the garden teaches is that our relationship to the planet need not be zero-sum, and that as long as the sun still shines and people still can plan and plant, think and do, we can, if we bother to try, find ways to provide for ourselves without diminishing the world.

- Michael Pollan, The Omnivore's Dilemma: A Natural History of Four Meals

Quote of the Day

There are plenty of difficult obstacles in your path. Don’t allow yourself to become one of them.

-Ralph Marston

Wisdom Of The Week

This feeling recurred as I moved about the neighborhood. I was seeing things I remembered seeing though I’d never seen them. A lifelong immersion in a writer’s work can do that to a reader, especially when the work is as vivid and particular as White’s. One morning I stopped at the general store in the crossroads town of Brooklin, near the post office where White used to pick up his mail every day, using a basket to carry home the bundles of unsolicited gifts and the stacks of fan letters, including, nearly 20 years before, those two breathless letters of mine.

- A walk through the home that inspired E. B. White's essays and stories

Quote of the Day

If I had my life to live over again, I would have made a rule to read some poetry and listen to some music at least once every week.

- Charles Darwin, The Autobiography of Charles Darwin, 1809–82

Quote of the Day

A key point to bear in mind: The value of attentiveness varies in proportion to its object. You’re better off not giving the small things more time than they deserve.

- Marcus Aurelius, Meditations

Quote of the Day

Quote of the Day

Quote of the Day

If the world were merely seductive, that would be easy. If it were merely challenging, that would be no problem. But I arise in the morning torn between a desire to improve the world and a desire to enjoy the world. This makes it hard to plan the day.

- E.B. White

Quote of the Day

From now on, he wrote, we must always take into account our knowledge that we can destroy ourselves at will, with all our history and perhaps life on earth itself. Nothing stops us but our own free choosing. If we want to survive, we have to decide to live. Thus, he offered a philosophy designed for a species that had just scared the hell out of itself, but that finally felt ready to grow up and take responsibility.

- Sarah Bakewell, At the Existentialist Café: Freedom, Being, and Apricot Cocktails

Quote of the Day

A good character is not life lived according to a rule, it is a life lived in balance.

- James Q. Wilson

Wisdom Of The Week

Consider the following thought experiment.

First case, one hundred persons go to a Casino, to gamble a certain set amount each and have complimentary gin and tonic –as shown in the cartoon in Figure x. Some may lose, some may win, and we can infer at the end of the day what the “edge” is, that is, calculate the returns simply by counting the money left with the people who return. We can thus figure out if the casino is properly pricing the odds. Now assume that gambler number 28 goes bust. Will gambler number 29 be affected? No.

You can safely calculate, from your sample, that about 1% of the gamblers will go bust. And if you keep playing and playing, you will be expected have about the same ratio, 1% of gamblers over that time window.

Now compare to the second case in the thought experiment. One person, your cousin Theodorus Ibn Warqa, goes to the Casino a hundred days in a row, starting with a set amount. On day 28 cousin Theodorus Ibn Warqa is bust. Will there be day 29? No. He has hit an uncle point; there is no game no more.
No matter how good he is or how alert your cousin Theodorus Ibn Warqa can be, you can safely calculate that he has a 100% probability of eventually going bust.

The probabilities of success from the collection of people does not apply to cousin Theodorus Ibn Warqa. Let us call the first set ensemble probability, and the second one time probability (since one is concerned with a collection of people and the other with a single person through time). Now, when you read material by finance professors, finance gurus or your local bank making investment recommendations based on the long term returns of the market, beware. Even if their forecast were true (it isn’t), no person can get the returns of the market unless he has infinite pockets and no uncle points. The are conflating ensemble probability and time probability. If the investor has to eventually reduce his exposure because of losses, or because of retirement, or because he remarried his neighbor’s wife, or because he changed his mind about life, his returns will be divorced from those of the market, period.

- Nassim Nicholas Taleb, The Logic of Risk Taking

Quote of the Day

• First, an A.I. system must be subject to the full gamut of laws that apply to its human operator.
• My second rule is that an A.I. system must clearly disclose that it is not human. 
• My third rule is that an A.I. system cannot retain or disclose confidential information without explicit approval from the source of that information.

- How to Regulate Artificial Intelligence

Quote of the Day

My method is different. I do not rush into actual work. When I get a new idea, I start at once building it up in my imagination, and make improvements and operate the device in my mind. When I have gone so far as to embody everything in my invention, every possible improvement I can think of, and when I see no fault anywhere, I put into concrete form the final product of my brain.

- Nikola Tesla