Complexity Explorer Santa Few Institute

Maximum Entropy Methods

Enroll

Lead instructor:

Your progress is not being saved! Enroll now or log in to track your progress or submit homework.
About the Tutorial:

Sometimes the best description of a system is one that acknowledges our ignorance. Maximum entropy methods provide a coherent framework for doing this in a rigorous fashion, and their success has been felt in everything from physics to ecology to machine learning.

In this tutorial, Simon DeDeo introduces MaxEnt through a series of examples, taking students from the basics of the subject up to its applications in science and engineering. Basic comfort with the use of probabilities, and familiarity with exponentials and logarithms, is required. By the end of the course, students will have a tool for modeling complex systems, and a new set of concepts for thinking about what models are meant to do in the first place.

Note that Complexity Explorer tutorials are meant to introduce students to various important techniques and to provide illustrations of their application in complex systems.  A given tutorial is not meant to offer complete coverage of its topic or substitute for an entire course on that topic.   

This tutorial is designed for more advanced math students.  Math prerequisites for this course are an understanding of calculus, partial derivatives, shannon entropy; basic probability.

Read Simon DeDeo's Q&A about his Maximum Entropy tutorial here.
About the Instructor(s):

Simon DeDeo is an assistant professor in the School of Social and Decision Sciences at Carnegie Mellon University, and external faculty at the Santa Fe Institute.

He received his A.B. from Harvard College in 2000, the Certificate of Advanced Study in Mathematics (Part III) from Cambridge University in 2001, and a Ph.D. in Astrophysics from Princeton University in 2005. Since 2010, he has been affiliated with the Santa Fe Institute, and has done research in cognition and social behavior. With colleagues and students at the Laboratory for Social Minds, he has undertaken quantitative studies ranging from innovation and conflict on Wikipedia to the creation of signals in the British legal system, the emergence of dominance hierarchies in the social animals, and the evolution and social dynamics of Anglo-American poetry, String Theory, and Harry Potter fan fiction. 

Read more about Simon and his work here. 

 
How to use Complexity Explorer
Enrolled students:

2,634
Prerequisites:

Calculus; Partial Derivatives; Shannon Entropy; Basic Probability
Like this tutorial?

10 Student
Ratings

Write your review Read reviews

Syllabus

  1. Maximum Entropy Methods (Simon DeDeo)
  2. A Simple Example: Waiting for a Taxicab
  3. The Maximum Entropy Method
  4. MaxEnt Applied to the Taxicab Example, Part 1
  5. MaxEnt Applied to the Taxicab Example, Part 2
  6. Review of MaxEnt
  7. A Real-World Example: Modeling the Open Source Ecosystem, Part 1
  8. Modeling the Open Source Ecosystem, Part 2
  9. Modeling the Open Source Ecosystem, Part 3
  10. A Second Real-World Example: Modeling Sears-Roebuck Catalog Prices, Part 1
  11. Modeling Sears-Roebuck Catalog Prices, Part 2
  12. Conclusion
  13. Homework