CSSS 2008 Beijing-Readings-Week-Three

From Santa Fe Institute Events Wiki

CSSS 2008 Beijing

Lee Altenberg

Lecture Notes

Additional Reading

  • Altenberg, L. , 1994. The Schema Theorem and Price's Theorem delves into the claims about schema processing as the source of power in genetic algorithms, and recasts the Schema Theorem (Holland 1975) by using Price's Theorem (1970). It is shown that the Schema Theorem says nothing about a GA's power, but a modification with a different measurement function produces a theorem about evolvability that is a local measure of GA power. The concept of rugged landscapes is also deconstructed in terms of operator-defined distance.

David Feldman

(Samuel) Qing-Shan Jia

Lecture Notes

Additional Reading

Susanne Still

Lecture Notes

July 14. A brief introduction to uses of information theory in machine learning

July 15. Optimal Causal Inference

Additional reading

You can find links to the following papers (and more) here.

  • C. E. Shannon: A mathematical theory of communication, Bell System Technical Journal, vol. 27, pp. 379-423 and 623-656, July and October, 1948.
  • K. Rose, "Deterministic Annealing for Clustering, Compression, Classification, Regression, and Related Optimization Problems," Proceedings of the IEEE, vol. 80, pp. 2210-2239, November 1998.
  • N. Tishby, F. C. Pereira, & W. Bialek, “The information bottleneck method”, in Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing, B Hajek & RS Sreenivas, eds, pp 368-377 (University of Illinois, 1999)

Please visit my website for my papers related to the material covered in the lectures:

  • S. Still and W. Bialek. “How many clusters? An information theoretic perspective”. 2004
  • S. Still, W. Bialek and L. Bottou. “Geometric Clustering using the Information Bottleneck method”. 2004.
  • S. Still and J. P. Crutchfield. Structure or Noise? 2007.
  • S. Still, J. P. Crutchfield and C. J. Ellison. Optimal Causal Inference. 2007.
  • S. Still. Statistical Mechanics approach to interactive learning. 2007.

Chris Wiggins

Lecture Notes

  • Bayes
    • History
    • What he said
    • What he didn't say
  • Bayes rule in Dynamics
    • Random walks
      • Pollen grains
      • Wall street
    • Chemical kinetics
    • Polymer Physics
  • Bayes rule in Statistics
    • Why we fit
    • Regularization
    • Latent variables / Mixture modeling
    • Model selection the Bayesian way
    • Bayesian density estimation / unsupervised learning
  • Alternatives to Bayes
    • Cross validation
    • pvalueology
    • Information theoretic methods

Slides for Chris's third lecture: Inferring and Encoding Graph Partitions . Also available here.

Additional Reading