Skip to main content

Advertisement

Neural conduction delay forces the emergence of predictive function in simulated evolution

Article metrics

Evidence from biological studies suggests that humans are able to predict the sensory consequences of their own actions [1]. Computational studies also demonstrate the advantage of systems that predict sensory consequences of actions over those that predict the value of actions alone [2]. But how could the ability to predict sensory consequences of actions have evolved? One solution suggested by [3] is that prediction mechanisms first evolved to deal with natural sources of delay. Delay is commonly considered to be a purely negative feature of real world systems; however, we argue that delay can actually encourage evolution of the prediction of sensory consequences. We hypothesize that increasing sensory delay to an evolving population of sensory-motor agents will increase reliance on internal prediction of sensory consequences.

To test our hypothesis we evolved populations of artificial neural networks at a complex control task (i.e. pole balancing, see figure 1) with varied neural conduction delay (Δt) between sensory neurons and input to the control network (see figure 2), which estimates the long term cost of applying a specific action. For top fitness networks, hidden unit activations were recorded as well as the true consequent sensory state during several evaluation trials. Each sensory variable was associated with the hidden unit that the sensory variable was maximally correlated with. Taking the average of these correlation values provides a measure of how well an agent can predict the sensory consequences of actions. We expected to find that increasing sensory delay also increases the average correlation measure described above.

The result of the experiment (summarized in figure 3) show that with no delay successful agents use a range of strategies, however, as delay increases successful strategies are forced to rely more and more on prediction of the next state to compensate for sensory delay. This seems surprising when considering that under conditions of no delay it is considerably easier to predict the next state than conditions with increased delay.

Although the common conception of delay is negative, sensory delay can direct natural selection to favor individuals that are better able to predict the sensory consequences of their actions.

Figure 1
figure1

Cart-Pole Balancing

Figure 2
figure2

Control network structure

Figure 3
figure3

Absolute correlation between hidden until activations and variables of the state at time t+Δt as delay increases.

References

  1. 1.

    Bullock D, Grossberg S, Geunther F: A Self-Organizing Neural Model of Motor Equivalent Reaching and Tool Use by a Multijoint Arm. Journal of Cognitive Neuroscience. 1993, 54: 408-435. 10.1162/jocn.1993.5.4.408.

  2. 2.

    Kuvayev L, Sutton RS: Model-based reinforcement learning with an approximate, learned model. Proceedings of the Ninth Yale Workshop on Adaptive and Learning Systems. 1996, 101-105.

  3. 3.

    Kwon J, Choe Y: Facilitating neural dynamics for delay compensation: A road to predictive neural dynamics?. Neural Networks. 2009, 22: 267-276. 10.1016/j.neunet.2009.03.011.

Download references

Author information

Correspondence to Timothy A Mann.

Rights and permissions

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Mann, T.A., Choe, Y. Neural conduction delay forces the emergence of predictive function in simulated evolution. BMC Neurosci 11, P62 (2010) doi:10.1186/1471-2202-11-S1-P62

Download citation

Keywords

  • Sensory Neuron
  • Successful Agent
  • Hide Unit
  • Sensory Consequence
  • World System