Complex Systems Summer School 2018-Projects & Working Groups

Complex Systems Summer School 2018 Home Surveys/Evaluations Logistics Payment Instructions Faculty Questions Faculty Publications Lecture Slides Participants Speed Dating ++ Schedule Preparation Background Readings Software Projects & Working Groups Project Group Meeting Schedule Sign Up to Meet with Project Liaison Presentation Sign Up Page Project Presentations Final Papers Tutorials After Hours Restaurant Guide Resource Guide Photos TShirts

Using Principles from Complex Systems in Thinking about AGI Development

AGI = Artificial General Intelligence, a catchphrase for "smarter-than-human" AI, a very misleading phrase which basically means algorithms which are generally capable of performing a wide range of tasks with high efficacy without being explicitly programmed to do each task.

For now, this is intentionally vague to keep open the various possibilities and gather together those who are interested. The project would move beyond current ML techniques, though, and either build on those techniques in significantly novel ways, propose new techniques, or consider from a theoretical standpoint how to design and train an agent (without specification of the implementation) which can perform a broad range of tasks "intelligently" and is aligned with human interests. An important focus is on ensuring alignment (doing what humans would want it to do), which is for various reasons quite hard to do both technically and philosophically.

There are two ways to use complex systems principles:

• In the design and training process of the algorithm
• In understanding how an algorithm will interact with the world around it

Specific project ideas:

• Building in an adaptive mechanism for an agent to adjust its input-output map as the dynamics of its environment change
• Using insights from various evolutionary processes to design a learning process that can produce an intelligent and aligned agent (either using existing AI techniques, or being implementation-agnostic and considering an arbitrary agent)

Feel free to add your name below, and any project ideas above! If we get a few interested people we can meet tonight or tomorrow.

Interested Participants:

• Luca Rade
• Nam Le

Neural style transfer in music styles via interacting agents

General idea: A) learn generative models of different music styles using neural networks. B) let these networks ('agents') interact and see what `fusion' music styles result.

Relevant papers: 1) neural style transfer for images (make images look like Van Gough paintings etc.) : https://tinyurl.com/ybpq5agm

2) neural nets for music: https://tinyurl.com/yb2qdqbq and http://imanmalik.com/cs/2017/06/05/neural-style.html

3) bunch of theories of how music styles are results of combination: https://tinyurl.com/y723ugyo

Novelty lies in having the a) multiple agents learn multiple styles independently then letting them exchange information in a meaningful way (probably the trickiest bit) and b) letting these fusion music styles evolve in a network etc. and see what "world-music" results at the end for example.

Details will come...

Thoughts?

• we could also use text corpora instead? Shakespeare etc.
• ...

Interested Participants:

• Yuki
• Vandana
• Xindi
• R Maria
• Kevin

Optimal representations of high dimensional data in deep learning and biological systems:

What is the best way for a system to represent very high dimensional data? For example, how should the retina encode visual stimuli in neuron firing patterns? How does the immune system encode the space of antigens it might encounter? In each case, it would not be feasible (or efficient) to create a unique tag for each input. Rather, the systems in question must decide which features in the stimuli are most relevant, and trade off between specificity and generality.

Along these lines, there are two more specific questions to investigate:

-It has recently been conjectured that the success of deep learning networks is related to their optimization of a specific informational quantity in each layer https://arxiv.org/abs/1710.11324. Unfortunately this paper is not very clearly written, but basically the idea is that when binning inputs into representations, the distribution of bin sizes should be given by a specific power law, which optimizes the aforementioned information measure. Do biological systems employ the same strategy? With access to the right data, this idea should be straightforward to test. For example, if we have a list of antibodies together with the set of antigens they react to, we can compute this quantity and see whether the antigen "bins" are indeed distributed according to the predicted power law.

-A diverse collection of biological systems that are faced with this task seem to be well-modeled by maximum entropy distributions, with a constraint on pairwise correlations and parameters (i.e. lagrange multipliers) set near a critical point https://arxiv.org/pdf/1012.2242.pdf. This has been applied to the previously given examples of the retina and the immune system, as well as flocking in birds. As far as I know, it is not yet known with certainty whether this kind of encoding scheme is optimal in some sense (like in the previous bullet), or if it is an artifact of our own inference methods, but I think the answer is interesting either way. An immediate question is, if these maximum entropy models are a powerful tool for humans to model high dimensional systems, might biological systems also be producing their own maximum entropy models of environmental variables? That is, are maximum entropy models with constraints on pairwise correlations optimal in some information-theoretic sense, which can be made precise? For example, would this be a particularly useful way to model the distribution of natural images one might encounter? While less straightforward than the previous bullet, I think these are questions well-suited to the skills of the people here, and I think we could make significant progress!

If anyone has expertise to offer, your feedback/participation would be very much appreciated! In particular, I think this project would greatly benefit from those of you that have knowledge in machine learning and biology (my own area is physics and information theory). Feel free to email me at e.stopnitzky@gmail.com

The Emergence and Evolution of Legal Systems as Pertaining to Water Distribution

‘’’General Idea’’’ There are numerous legal systems that have been identified, broadly categorized into large families – Common Law (Anglosphere and Commonwealth nations), Civil Law (Romance Language nations, Germany, China), Islamic law (most Muslim nations), Customary Law (India, sub-Saharan Africa). More importantly, most nations do not purely lie in one category, but tend to combine elements of multiple systems, either due to merging (i.e. German law combining Germanic tradition with Civil traditions), or through subsidiarity (i.e. Louisiana having Napoleonic law, despite being in a Common Law nation). We are interested in determining how these legal systems by nations and states emerged, influenced each other, and interact over national boundaries.
This is an immense task, as so to scope it, one idea has been to limit this project to laws pertaining to water distribution. This is of particular interest when looking at states of nations that have different legal systems, such as Louisiana in the U.S., Quebec in Canada, and Scotland in the U.K. For international interactions, sub-Saharan African nations might also be of value in assessing, as many nations border nations with different legal systems, and water is often a scares resource in these areas.
If anyone has interest in this topic, and/or expertise in either legal systems or water distribution, feel free to sign up or discuss.
‘’’Recommended Papers’’’ ‘’’Interested Participants’’’ 1. Kevin Comer