CSSS 2008 Beijing-Readings-Week-Three

CSSS 2008 Beijing Home Faculty Participants Highlights Photos Schedule Readings and Lecture Notes Week One Week Two Week Three Week Four Projects T-Shirt Chinese Visa Information Travel Details and Logistics Final Papers

Lee Altenberg

Lecture Notes

• Lecture 1: Introduction to Evolutionary Computation (7 MB)

• Lecture 2: Mathematical Analysis of Evolutionary Algorithms

• Lecture 3: Higher Order Evolutionary Phenomena (8MB)

Additional Reading

• Altenberg, L. , 2004. Open Problems in the Spectral Analysis of Evolutionary Dynamics presents a mathematical framework for evolutionary optimization and some of its unsolved problems.

• Altenberg, L. , 1994. The Schema Theorem and Price's Theorem delves into the claims about schema processing as the source of power in genetic algorithms, and recasts the Schema Theorem (Holland 1975) by using Price's Theorem (1970). It is shown that the Schema Theorem says nothing about a GA's power, but a modification with a different measurement function produces a theorem about evolvability that is a local measure of GA power. The concept of rugged landscapes is also deconstructed in terms of operator-defined distance.

• Erik van Nimwegen, James P. Crutchfield, and Martijn Huynen. Neutral Evolution of Mutational Robustness. Proceedings of the National Academy of Science U.S.A. 96:9716-9720 (1999).

• Altenberg, L. , 2004. Modularity in Evolution: Some Low Level Questions deconstructs the concept of modularity in terms of spaces of variation, and discusses properties needed for modularity to enhance evolvability.

• Erik van Nimwegen and James P. Crutchfield. Metastable Evolutionary Dynamics: Crossing Fitness Barriers or Escaping via Neutral Paths? Bulletin of Mathematical Biology 62:5:799-848 (Sep 2000)

• Lauren Ancel Meyers, Fredric D. Ancel, Michael Lachmann. 2005. Evolution of Genetic Potential. PLoS Computational Biology 1(3): e32.

• Evolutionary Art at Electric Sheep

David Feldman

• Thoughts on Research Topics, Questions, and Problems
• Tips and Advice for giving Effective Research Presentations

(Samuel) Qing-Shan Jia

Lecture Notes

• Ordinal Optimization: Soft Optimization for Hard Problems
• How does Spare Capacity and Topology Affect the Security of Resource Networks?

Additional Reading

• An Explanation of Ordinal Optimization. Ho. Information Sciences 113, 169-192 (1999).
• Universal Alignment Probabilities and Subset Selection for Ordinal Optimization. Lau and Ho. Journal of Optimization Theory and Applications 93:3. 455-489 (1997).
• How Topology Affects Security: An Upper Bound of Electric Power Network Security. Jia and Zhao.
• How Much Spare Capacity is Necessary for the Security of Resource Networks? Zhao, Jian and Cao. Physica A 373, 861-873 (2007).

Susanne Still

Lecture Notes

July 14. A brief introduction to uses of information theory in machine learning

July 15. Optimal Causal Inference

Additional reading

You can find links to the following papers (and more) here.

• C. E. Shannon: A mathematical theory of communication, Bell System Technical Journal, vol. 27, pp. 379-423 and 623-656, July and October, 1948.

• K. Rose, "Deterministic Annealing for Clustering, Compression, Classification, Regression, and Related Optimization Problems," Proceedings of the IEEE, vol. 80, pp. 2210-2239, November 1998.

• N. Tishby, F. C. Pereira, & W. Bialek, “The information bottleneck method”, in Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing, B Hajek & RS Sreenivas, eds, pp 368-377 (University of Illinois, 1999)

• Books:
  • D. MacKay, "Information Theory, Inference and Learning Algorithms"
  • T. M. Cover and J. A. Thomas, "Elements of Information Theory", Wiley.

Please visit my website for my papers related to the material covered in the lectures:

• S. Still and W. Bialek. “How many clusters? An information theoretic perspective”. 2004

• S. Still, W. Bialek and L. Bottou. “Geometric Clustering using the Information Bottleneck method”. 2004.

• S. Still and J. P. Crutchfield. Structure or Noise? 2007.

• S. Still, J. P. Crutchfield and C. J. Ellison. Optimal Causal Inference. 2007.

• S. Still. Statistical Mechanics approach to interactive learning. 2007.

Chris Wiggins

Lecture Notes

• Bayes
  • History
  • What he said
  • What he didn't say
• Bayes rule in Dynamics
  • Random walks
    • Pollen grains
    • Wall street
  • Chemical kinetics
  • Polymer Physics
• Bayes rule in Statistics
  • Why we fit
  • Regularization
  • Latent variables / Mixture modeling
  • Model selection the Bayesian way
  • Bayesian density estimation / unsupervised learning
• Alternatives to Bayes
  • Cross validation
  • pvalueology
  • Information theoretic methods

Slides for Chris's third lecture: Inferring and Encoding Graph Partitions . Also available here.

Additional Reading

• wikipedia (be careful! there's likely to be at least one error per page):
  • http://en.wikipedia.org/wiki/Thomas_Bayes
  • http://en.wikipedia.org/wiki/Bayes'_theorem
  • http://en.wikipedia.org/wiki/Gamma_function
  • http://en.wikipedia.org/wiki/Stirling%27s_approximation
  • http://en.wikipedia.org/wiki/Method_of_steepest_descent#The_idea_of_Laplace.27s_method
  • http://en.wikipedia.org/wiki/Gaussian_integral
• bayes's biography: http://www-gap.dcs.st-and.ac.uk/~history/Mathematicians/Bayes.html
• papers:
  • bayes's original essay: http://www.stat.ucla.edu/history/essay.pdf
  • mackay's thesis, Chapter 2. (take a look at his book, too below) http://www.inference.phy.cam.ac.uk/mackay/PhD.html
  • nice Occam’s razor paper by Zoubin Ghahramani http://learning.eng.cam.ac.uk/zoubin/papers/05occam/occam.pdf
  • schwartz's paper http://www.math.tau.ac.il/~yekutiel/MA%20seminar/Schwarz%201978.pdf
  • those of you trained in statistical field theory will like this paper also http://arxiv.org/abs/adap-org/9601001
• mackay's book: http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
• references to books not available online:
  • http://research.microsoft.com/~cmbishop/prml/