Why Feature Engineering, Domain & Holistic Knowledge Is Important In Modeling & Life

E.O.Wilson called this Consilience: The Unity of Knowledge: 

Still, if history and science have taught us anything, it is that passion and desire are not the same as truth. The human mind evolved to believe in the gods. It did not evolve to believe in biology. Acceptance of the supernatural conveyed a great advantage throughout prehistory when the brain was evolving. Thus it is in sharp contrast to biology, which was developed as a product of the modern age and is not underwritten by genetic algorithms. The uncomfortable truth is that the two beliefs are not factually compatible. As a result those who hunger for both intellectual and religious truth will never acquire both in full measure.

[---]

The greatest challenge today, not just in cell biology and ecology but in all of science, is the accurate and complete description of complex systems.

Robert Rubin in his book In an Uncertain World: Tough Choices from Wall Street to Washington explains why brute force math based modeling is pure bullshit: 

Sound decisions are based on identifying relevant variables and attaching probabilities to each of them. That's an analytic process but also involves subjective judgements. The ultimate decision then reflects all of this input, but also instinct, experience, and 'feel'. All the time bearing in mind that reality is always more complex than concepts and models.

A true probabilistic view of life quickly leads to the recognition that almost all significant issues are enormously complex and demand that one delve into those complexities to identify the relevant considerations and the inevitable trade-offs. With an enormous number of competing considerations, the key to reaching the best possible decision is to identify all of them and decide what odds and import to attach to each.

In order to live sanely in an uncertain world without believing in magic; we need to embrace a probabilistic view of the world plus constantly update that probability via a bayesian loop. 

We humans are the victims of an asymmetry in the perception of random events. We attribute our successes to our skills, and our failures to external events outside our control, namely to randomness. 

- Nassim Nicholas Taleb