Inductive Inference and Solomonoff Induction

by jseldred

About me: My name is Jeffrey Eldred and (right now) I am a physics PhD student conducting research at Fermilab in accelerator physics. I’ve always had an interest in philosophical topics and for the last five years I’ve written for a Q&A site AllExperts: Atheism.


Presentation on Solomonoff Induction: I presented on Inductive Inference and Solomonoff Induction for the fledgling Fermi Philosophical Society yesterday (April 10th 2014). A video of my presentation can be found here and the slides I’m using can be found here. The talk was well-received and sparked many interesting discussions that may be the subject of future talks.


References: I’ve put together a list of references to explore the mathematics and philosophy presented in the talk more deeply.

This is a description of Solomonoff Induction for a general audience prepared by the applied philosophy website LessWrong. It was something of an inspiration for this talk and (by necessity) it matches the topics covered in my talk quite well. They also have a wiki which writes out the mathematical result from Solomonoff Induction. I would also recommend a talk by Eliezer Yudkowsky (founder of Less Wrong) which explains how parsimony relates to precise analytical thinking and how the humans are naturally biased towards the unparsimonious.

If you’d like to go straight to the source, one can read Solomonoff original writing as he presented Solomonoff Induction for the first time: Part I and Part II.

Bayes Theorem was an important topic covered in my talk. Bayes Theorem is useful but unintuitive. Here is the landmark psychological study showing a failure in Bayesian reasoning among physicians. Richard Carrier, a Biblical and statistical scholar, has been an outspoken advocate of atheism as well as the application of Bayes Theorem to historical scholarship. This is a talk by Richard Carrier – the first half focuses on Bayes Theorem and how to use it; the second half uses a Bayesian argument against the existence of God (and it particular, special pleading).

If I had more time in my talk, I would have covered Edwin Jaynes’ contribution to probability in greater depth. Its in one of my presentation’s backup slides. But I highly recommend reading Jaynes’ original Information Theory and Statistical Mechanics. It explores the relationship between the principle of indifference and identical microstates in what has come to be known as the Maximal Entropy Principle. This important work gives insight into both fields of information theory and statistical mechanics.

Principle of Indifference was also a key part of the talk. I think Wikipedia’s article on Bertrand’s Paradox (and how to resolve it) is an excellent case study how the principle of indifference might be extended into continuous geometric space. I also have a backup slide about this.

Laplace was the one who really popularized both Bayes Theorem and the Principle of Indifference. You can read a free full translation of his Philosophical Essay on Probabilities here, but there is also a more modern translation.

Finally I will leave you with a list of fulltext links to other philosophical works references in this talk:

David Hume’s Dialogue’s Concerning Natural Religion:

Introduction to Epistemological Pluralism.

Stephen Jay Gould’s Nonoverlapping Magisteria.

David Hume’s Enquiry Concerning Human Understanding:

Ludwig Wittgenstein’s Philosophical Investigations.

Douglas Hofstadter’s Godel, Escher, Bach: Eternal Golden Braid