CSSS 2008 Neuroscience Working Group

This page describes work on a project related to neuroscience issues for the 2008 CSSS.

What questions do we want to answer?

The overarching question I'd like to address is:

How do the decision-making criteria of the indivual affect the behavior of the group?

There are a lot of more specific questions we could pose related to this; for the first meeting or two, I'd like to zoom in on asking things in a more constrained way.

An initial idea for a project could be this:

Recent neuroscience studies (cited below) have modeled the representation of expected and experienced value in a dynamic foraging environment. E.g., we can model a primate facing a set of choices of where to forage; different locations have a different likelihood of providing food (=reward). Because the environment is dynamic, the reward likelihood of each location can change unpredictably in time. Corrado and colleagues propose a linear/nonlinear/Poisson model for this sort of decision making, which predicts well physiological data from a parietal region (LIP) in monkeys engaged in this task.

This model gives us a way to estimate the value associated with different locations, based on one monkey's personal experience. I'd like to add to this a representation of how monkeys (or other, more abstract agents) combine this information with information about what other monkeys are doing. The second part of the model would be based/inspired by the models Iain Couzin described on group behavior. The exact details can be open to discussion, but we could look to construct a simple model where each monkey looks at other monkeys in a given radius, sees which ones are succeeding at getting reward, and updates the estimated value of those locations, with a temporal decay signal.

So, I propose we construct a value representation with these two terms -- one reflecting an agent's individual experience and memory of reward, and one representing the agent's perception of how successful nearby agents are. We would add a weighting factor to adjust between these terms. We should have some expectations about the behavior of the set of agents at the extreme values of these weights: e.g., when the weight for information coming from other agents is turned to zero, the model should reproduce the behavior reported in the Corrado et al. papers.

Connection with the multi-armed Bandit problem

Speaking with Anthony and Nish, it turned out that the foraging task in which the matching law gives the general principle of primate behaviour resembles very much the 2-armed bandit problem (enwiki page, not a very high quality description). This is a traditional problem in Machine learning and it would be interesting to see if there are connection with the learning algorithms used to have an optimal strategy of reward. Nish has the code for TD-learning already implemented in Scilab. Giovanni

What is the relevant background?

Articles on the neuroscience of decision-making:

• short review article outlining general approaches towards modeling decision-making in neuroscience: a good place to start
• Science paper presenting the initial experiments on a simplified foraging task, and trying to understand neural correlates of perceived "value"
• theory paper explaining the "LNP" (linear-nonlinear-Poisson) model used to model decision-making processes
• Rushworth and Behrens 2008 Review article on work in cingulate and parietal cortex of humans by the Rushworth group at Oxford.

How shall we answer them?

My (Rory's) only criterion: all code will be in MATLAB. No disrespect for other languages: I think Python's great and am learning it, and NetLogo is convenient; but I know for a fact that things will get done much faster if I can code in MATLAB. I'm just that kinda guy. :)

Logistics: when and where do we meet?

First meeting is at 9pm in the Lower Dorm Common Room, Friday 6/13.

Hey I've waited a bit in common room but nobody showed up. Maybe we can meet tomorrow or wait for Nish to get back from the gran canyon Giovanni

People: Who's interested?

Please sign below if you'd like to work on this with me. I'll make an email list with names from these people. (You don't have to give me your address here; I can figure it out.)

1. Rory_Sayres
2. Giovanni
3. Chris