For citation purposes: Chen LL, Tehovnik EJ, May PJ. Does the brain compute the future or the present? OA Neurosciences 2014 Apr 10;2(1):7.

Short communication

 
Systems

Does the brain compute the future or the present?

L Chen1*,2*, E Tehovnik5, P May2,3
 

Authors affiliations

(1) University of Mississippi Medical Center, Department of Otolaryngology, Jackson, MS 39216, USA

(2) University of Mississippi Medical Center, Department of Ophthalmology, Jackson, MS 39216, USA

(3) University of Mississippi Medical Center, Department of Neurobiology and Anatomical Sciences, Jackson, MS 39216, USA

(4) University of Mississippi Medical Center, Department of Neurology, Jackson, MS 39216, USA

(5) Brain Institute, UFRN, Natal, RN, 59056-450, Brazil

* Corresponding author Email: lchen2@umc.edu

Abstract

Introduction

Decades of studies have suggested that the visuomotor system possesses the capacity to predict the locations of targets based on their past trajectories. Whether this capacity is used to guide rapid saccadic eye movements remains largely unknown. A recent study employed an oculomotor version of the sport of baseball to test two competing hypotheses: the dual-drive and single-drive theories. The findings were consistent with other recent studies, supporting the notion that the saccadic system receives continuous updates of the target representation from the pursuit system. This occurs even if the target is temporarily occluded. In addition, they showed that the observed intercepting saccade is aimed at the computed location of the target at the moment of saccade completion. It is noteworthy that saccadic eye movement accuracy corresponds to the spatial and temporal range of the visual sensory system, such that the perception of the target and the production of the action are in the right place (on target) at the right time (on time). Based on the implications of these recent findings, we suggest that the saccade generator behaves like an embedded real-time processing unit that updates the perceptual frame of reference and compensates for delays. The aim of this paper is to review the data and literatures pertinent to this issue.

Short Communication

A recent study employed an oculomotor version of the sport of baseball to test two competing hypotheses: the dual-drive and single-drive theories. The findings were consistent with other recent studies, supporting the notion that the saccadic system receives continuous updates of the target representation from the pursuit system. This occurs even if the target is temporarily occluded. In addition, they showed that the observed intercepting saccade is aimed at the computed location of the target at the moment of saccade completion. It is noteworthy that saccadic eye movement accuracy corresponds to the spatial and temporal range of the visual sensory system, such that the perception of the target and the production of the action are in the right place (on target) at the right time (on time). The aim of this paper is to review the data and literatures pertinent to this particular issue.

Conclusion

Based on the implications of the recent findings, we suggest that the saccade generator behaves like an embedded real-time processing unit that updates the perceptual frame of reference and compensates for delays.


Introduction

Imagine a fielder attempting to catch a baseball flying into his field at a speed of more than 100 mph. He runs as fast as he can, perhaps taking his eyes off the ball for a moment to ascertain the location of other fielders, then opens his glove in the right place at the right time to make the catch. This demonstrates the athlete’s amazing capacity for sensorimotor control. Every day, we perform similar routines, such as grabbing a full cup of hot coffee being handed to us. These tasks are so deceivingly simple that we take them for granted. In fact, they require a precise sensorimotor control no less complicated than catching a fly ball, albeit at a slower pace. For decades, neuroscientists have tried to find out how sensorimotor signals are updated in the brain to allow effective interaction with a changing world[1,2,3,4,5,6]. Moreover, recent developments of brain-machine interfaces are aimed at achieving the same capacity[7]. Amazingly, despite decades of studies [8,9,10,11], exactly how the brain accomplishes this precise sensorimotor control remains elusive. Below, we will review the data and literatures pertinent to this issue.

Short Communication

The authors have referenced some of their own studies in this short communication.  The protocols of these studies have been approved by the relevant ethics committees related to the institution in which they were performed. Animal care was in accordance with the institution guidelines.

Baseball Interception Game

A recent investigation by Fleuriet and Goffart [12] has taken a new approach based on the oculomotor perspective. The authors trained nonhuman primates to play an oculomotor version of baseball, i.e., to intercept a moving visual target with their eye movements (Figure 1A). Normally, when animals pursue a moving target, they make a rapid eye movement (saccade) to intercept the target, followed by a pursuit movement that matches the speed and direction of the target. In this variant, the animals were trained to fixate at the centre of the screen. After some time, the fixation point was turned off and a moving target appeared at a different position. However, the animal only had 150 ms to view the target’s motion before it was hidden from the animal’s view. The hidden target followed its original trajectory and reappeared at a new location 150 ms later to continue its course. As one might imagine, the animals disengaged their fixation and successfully obtained the target with an intercepting saccade (Figure 1A, a), and then followed it until they received their reward. This cognitive capability of knowing where an occluded moving target will reappear is called object permanence, and it has been widely documented since Piaget[13].

Baseball interception paradigm. Subjects begin with fixating at an initial position (glove) and attempt to intercept a moving target (baseball) which is viewed for 150 ms (A) or 300 ms (B) before occlusion. The flying baseball continues its original trajectory and reappears after the occluding time expires. The observed saccades result in catching the ball (a) or not catching the ball (h) primarily depending on the occlusion duration. Pursuit eye movement (f, i) follows, if the saccade fails to catch the ball. The nervous system shifts the position of the glove when artificially stimulated (b). If the originally programmed saccade were made, the result would be inaccurate (d). Instead, a compensatory saccade (e) is made. If there was no efference copy compensation and no input predicting the occluded motion of the target, the result would be (c). If the brain behaved predictively into the future, the saccade should always catch the ball (g) regardless of the occlusion duration. C and D: Key elements of the dual-drive (C) and single-drive (D) theories that account for the observed saccadic interception. CB: cerebellum, EC: efference copy, FOR: fastigial oculomotor region, SC: superior colliculus.

What is new in this study is that the positions of the eyes were perturbed (Figure 1A, b), as soon as the target disappeared from the animal’s view, by electrically stimulating the superior colliculus (SC) for 30 ms. The SC is an oculomotor structure critical for making saccadic eye movements. It computes target location relative to the current eye position, i.e., target displacement (direction and amplitude). Electrical stimulation in the colliculus overrides impending motor commands, bringing the eyes to a new position, topographically encoded on the collicular map[14,15,16]. The authors[12] reasoned that, if the animal relied solely on the location of the last-seen target relative to its eye position before stimulation, i.e., without taking into account the change of eye position resulting from collicular stimulation, the animal would miss the target by the amount equivalent to the stimulation-elicited saccade (Figure 1A, c). This result can only be realized if the system cannot compensate for the saccades made due to collicular stimulation. However, Mays and Sparks[17] had shown that when the impending saccade is directed from a given initial position to a last-viewed (but no longer visible) target, injecting current into the SC evokes a stimulation-evoked saccade followed by a compensatory saccade that accurately acquires the target. This suggests that a copy of the motor signal produced by the stimulation (efference copy, or termed the resettable integrator in models[18], Figure 1) is used by the system to take into account the stimulation-evoked displacement and remap the required saccade vector. Consequently, the colliculus is able to drive a successful compensatory saccade. The difference between the paradigms is that the target was stationary in Mays and Sparks’ experiments[17].

It has been assumed that the saccadic target selection is totally segregated from the motion/pursuit pathway[19,20,cf,8], and so the colliculus, while aware of the new eye position, would not be able compensate for the new relative position of the reappearing target. Interestingly, the animals in Fleuriet and Goffart’s experiment did make an accurate intercepting saccade (Figure 1A, e). This strongly suggests the estimated motion of the occluded target must be incorporated into the generation of the saccade.

Dual-drive theory and single-drive theory

The above finding is relevant to the predictions of a popular oculomotor control theory, the dual-drive hypothesis[19, 20]. This hypothesis includes the cerebellum (CB) as the second drive in addition to the SC’s drive[cf. 21, 22] and assumes that the saccadic and motion/pursuit systems are totally separate (Figure 1C). The SC acts as a gateway for the saccadic pathway, encoding a static (primarily visible) target representation. In contrast, the cerebellum computes visual motion signals based on the history of target trajectory, such that the target no longer needs to be visible[23]. The motion signals are converted for use as a saccadic drive at one of the cerebellum’s outputs, the fastigial oculomotor region (FOR). This drive and the SC’s drive are summed at the level of the premotor neurons in the saccade generator. That is how the dual-drive hypothesis[19,20] accounts for saccades to a moving target.

Note that in this model the SC produces the strongest drive at saccade onset when it drives the eyes toward the location of the target at saccade initiation[19,20]. The cerebellum contributes minimally at this point[20,]. The FOR modifies the saccadic trajectory during the decelerating phase of the saccade, to produce an accurate saccade to a moving target[21,22,24,25]. That is, with or without collicular perturbation, both the timing and relative location of targeting saccade initiation would be changed with respect to the movement calculation produced by the cerebellum. According to the dual drive model, the trajectory of the saccade would be modified midflight as a result of the late-coming cerebellar modulation, hence should curve toward where the target was initially occluded (i.e., the interception of vectors d and f in Figure 1A). Interestingly, this prediction was not observed. The intercepting saccades were largely straight (i.e., vector e in Figure 1A) or curved slightly away from where the target was initially occluded[Figure,2B,in,12]. This implies that the pursuit movement prediction system has access to the saccade system at saccade initiation before the cerebellum was fully engaged in modifying the saccadic trajectory. This also suggests that the target prediction information is available to the superior colliculus.

Another core question driving the debate is: How does the brain estimate target position during occlusion? There are observations suggesting that the brain may adopt a predictive strategy to cope with the uncertainty resulting from target occlusion[26,27,28]. The occluded target displacement (in degrees) can be estimated as the multiplicative product of a motion speed estimate (in degrees/second) and the occluding duration (in seconds). The target position before occlusion is known. The dual drive model simply sums the two movement signals: the past signal generated before target occlusion and the future signal, a prediction based on trajectory (Figure 1C). As we have discussed above, the blocked processing feature of the dual-drive hypothesis cannot explain satisfactorily the tell-tale of the observed saccade: the lack of curving in the saccadic trajectory.

The alternative is that the saccadic system must have access to the prediction, or be able to make the prediction itself. This alternative is essentially a single-drive (Robinsonian) saccade model in which saccades can be driven by the representation of a moving target at the level of the SC or the FEF (Figure 1D). Recent studies in the saccadic region of the frontal eye field (FEF) support this possibility, because the target representation at the level of the FEF is updated continuously, even if the target is occluded and is in motion[8,29]. Presumably this information should then be available to the colliculus via corticotectal input.

Future vs. present

Clearly the brain is in the business of making predictions[1,3,8,30,31,32,33,34], but are predictions of the future being used in making the compensatory saccade? Fleuriet and Goffart[12] varied the duration of target occlusion from 150 to 300 ms (Figure 1B) to test this question. Under these conditions, the animal often made a compensatory saccade (Figure 1B, h) that fell short of the location of the reappearance of the moving target (Figure 1B, g endpoint). These saccades were often followed by a pursuit-like eye movement or a corrective saccade[35] along the target trajectory (Figure 1B, i), as if the system were tracking a virtual moving target. These observations were taken by the authors[12] to argue that the eye movement was not generated in response to a predicted location of the target in the future. Rather, the eye movement was generated in response to where the occluded target is computed to be at present.

The authors developed a NOW (or here-and-now) theory[12,36] out of these and other observations. For instance, the displacement errors between the saccadic endpoints and the occluded target positions remained relatively constant, regardless of the total elapsed time from the onset of the stimulation-evoked saccades to the offset of the intercepting saccade[Figure6,in,12]. This is consistent with the finding that the amplitudes of intercepting saccades were linearly correlated with the target displacements[Figure]3 in 12. These observations all support the notion that the animals attempted to intercept the occluded target at its present computed location. This suggests that moving target representation is continuously updated within the saccadic system even when it is occluded prior to or during saccades[3,8,29,31,32,33]. The continuously evolving desired eye position then drives the saccade generator to respond accordingly (Figure 1D). Thus, the system is not actually looking into the future, it is directed at acquiring the target at its present location. Where this computation occurs and how it accesses the colliculus are open questions.

The NOW hypothesis provokes a number of questions. One is: How does the brain compute the present location of the target, given that signal processing of sensory inputs takes time[37,38,39]. No matter how fast the brain computes, by the time the processed signal arrives, it is guaranteed to be outdated[40]. The same is true for proprioceptive information on current position[2,41]. Consequently, if based purely on current sensory inputs, the movement triggered is bound to lag behind that of the intended target[42,43]. Additionally, the system has to compensate for the lag between neural activity and actual movement to attain the target. These inherent limitations necessitate a mechanism that is fast and runs its own pre-determined programs[40]. One possible engineering solution, called embedded real-time processing, runs its own pre-programmed controls that compensate for the system’s processing delay[44]. The saccade generator in the brainstem possesses many characteristics that are compatible with this notion[11,15,16,45]. This possibility deserves to be thoroughly explored, if we are to understand how motor systems work under real-world conditions.

To make progress in this regard, we may also need to give the concept of present (or here-and-now) a clearer definition in the context of the nervous system. The brain is limited by its own computational power, such that stimuli falling within the spatial-temporal processing limit are perceived as on target (HERE) and on time (NOW). For instance, the visual system detects flicker only when the refresh rate of a monitor is set below 60 Hz, giving the visual system a temporal resolution of 16-ms[46,47]. Thus, the conscious experience of being on target may be determined by the visual system’s resolution. For instance, a saccade to a target moving at 20 deg/sec would still be considered on target, even if the intercepting saccade missed the target by ~1/3 of a degree (0.32 deg = 20 deg/sec x 16 ms). Interestingly, these numbers appear to be in close agreement with observations in the target interception experiment[Figure] [8,in,36]. Thus, the saccade generator works within the predictive perceptual frame of reference[3,31,32,33]. Within such a tight spatial and temporal window, it is difficult to tell whether it is the sensory or motor system that drives the perception of being on target or on time[1,9,10,40].

Conclusion

Based on the above review, we suggest that the saccade generator behaves like an embedded real-time processing unit that updates the perceptual frame of reference and compensates for delays.

Acknowledgement

We are thankful for the stimulating discussions with Dr. Jerome Fleuriet. This work was supported by EY016710 (LLC), CNPq #300690/2012-6 (EJT), and EY014263 (PJM).

Conflict of interests

None declared.

Competing interests

None declared

References

1. Graf AB, Andersen RA. Inferring eye position from populations of lateral intraparietal neurons. Elife. 2014;3:e02813.

2. Xu BY, Karachi C, Goldberg ME. The postsaccadic unreliability of gain fields renders it unlikely that the motor system can use them to calculate target position in space. Neuron. 2012 Dec;76:1201-09.

3. Duhamel JR, Colby CL, Goldberg ME. The updating of the representation of visual space in parietal cortex by intended eye movements. Science. 1992 Jan;255:90-92.

4. Graziano MS. Is reaching eye-centered, body-centered, hand-centered, or a combination? Rev Neurosci. 2001;12:175-85.

5. Snyder LH, Batista AP, Andersen RA. Coding of intention in the posterior parietal cortex. Nature. 1997 Mar;386:167-70.

6. Kawato M. Internal models for motor control and trajectory planning. Curr Opin Neurobiol. 1999 Dec;9:718-27.

7. Tehovnik EJ, Woods LC, Slocum WM. Transfer of information by BMI. Neuroscience. 2013 Dec;255:134-46.

8. Ferrera VP, Barborica A. Internally generated error signals in monkey frontal eye field during an inferred motion task. J Neurosci. 2010 Sep;30:11612-23.

9. Sommer MA, Wurtz RH. Brain circuits for the internal monitoring of movements. Annu Rev Neurosci. 2008;31:317-38.

10. Holst E, Mittelstaedt H. The reafference principle: Interaction between the central nervous system and the peripheral organs. In: Selected Papers of Erich von Holst: The Behavioural Physiology of Animals and Man, pp 39-73 London: Methuen. 1950.

11. Robinson DA. Oculomotor control signals. In: Basic Mechanisms of Ocular Motility and Their Clinical Implications (Lennerstrand G, Bach-y-Rita P, eds), pp 337-74: Oxford: Pergamon Press. 1975.

12. Fleuriet J, Goffart L. Saccadic interception of a moving visual target after a spatiotemporal perturbation. J Neurosci. 2012 Jan;32:452-61.

13. Piaget J. The construction of reality in the child. New York: Norton, 1954.

14. Robinson DA. Eye movements evoked by collicular stimulation in the alert monkey. Vision Res. 1972 Nov;12:1795-808.

15. Sparks DL, Mays LE. Signal transformations required for the generation of saccadic eye movements. Annu Rev Neurosci. 1990;13:309-36.

16. Gandhi NJ, Katnani HA. Motor functions of the superior colliculus. Annu Rev Neurosci. 2011;34:205-31.

17. Mays LE, Sparks DL. Saccades are spatially, not retinocentrically, coded. Science. 1980 Jun;208:1163-65.

18. Jurgens R, Becker W, Kornhuber HH. Natural and drug-induced variations of velocity and duration of human saccadic eye movements: evidence for a control of the neural pulse generator by local feedback. Biol Cybern. 1981;39:87-96.

19. Keller EL, Gandhi NJ, Weir PT. Discharge of superior collicular neurons during saccades made to moving targets. J Neurophysiol. 1996 Nov;76:3573-77.

20. Quaia C, Lefevre P, Optican LM. Model of the control of saccades by superior colliculus and cerebellum. J Neurophysiol. 1999 Aug;82:999-1018.

21. Noda H, Murakami S, Warabi T. Effects of fastigial stimulation upon visually-directed saccades in macaque monkeys. Neurosci Res. 1991 Apr;10:188-99.

22. Ohtsuka K, Sato H, Noda H. Saccadic burst neurons in the fastigial nucleus are not involved in compensating for orbital nonlinearities. J Neurophysiol. 1994 May;71:1976-80.

23. Krauzlis RJ, Lisberger SG. Visual motion commands for pursuit eye movements in the cerebellum. Science. 1991 Aug;253:568-71.

24. Goffart L, Chen LL, Sparks DL. Deficits in saccades and fixation during muscimol inactivation of the caudal fastigial nucleus in the rhesus monkey. J Neurophysiol. 2004 Dec;92:3351-67.

25. Goffart L, Chen LL, Sparks DL. Saccade dysmetria during functional perturbation of the caudal fastigial nucleus in the monkey. Ann N Y Acad Sci. 2003 Oct;1004:220-28.

26. Barnes GR, Barnes DM, Chakraborti SR. Ocular pursuit responses to repeated, single-cycle sinusoids reveal behavior compatible with predictive pursuit. J Neurophysiol. 2000 Nov;84:2340-55.

27. Bennett SJ, Orban de Xivry JJ, Barnes GR, Lefevre P. Target acceleration can be extracted and represented within the predictive drive to ocular pursuit. J Neurophysiol. 2007 Sep;98:1405-14.

28. Heinen SJ. Oculomotor hide and seek: pursuing an accelerating target behind an occluder. Focus on "target acceleration can be extracted and represented within the predictive drive to ocular pursuit". J Neurophysiol. 2007 Sep;98:1073-74.

29. Barborica A, Ferrera VP. Estimating invisible target speed from neuronal activity in monkey frontal eye field. Nat Neurosci. 2003 Jan;6:66-74.

30. Costa VD, Averbeck BB. Looking into the future. Elife. 2014;3:e03146.

31. Mirpour K, Bisley JW. Anticipatory remapping of attentional priority across the entire visual field. J Neurosci. 2012 Nov;32:16449-57.

32. Shin S, Sommer MA. Division of labor in frontal eye field neurons during presaccadic remapping of visual receptive fields. J Neurophysiol. 2012 Oct;108:2144-59.

33. Jonikaitis D, Szinte M, Rolfs M, Cavanagh P. Allocation of attention across saccades. J Neurophysiol. 2013 Mar;109:1425-34.

34. Krauzlis RJ, Nummela SU. Attention points to the future. Nat Neurosci. 2011 Feb;14:130-31.

35. Bennett SJ, Barnes GR. Timing the anticipatory recovery in smooth ocular pursuit during the transient disappearance of a visual target. Exp Brain Res. 2005 May;163:198-203.

36. Fleuriet J, Hugues S, Perrinet L, Goffart L. Saccadic foveation of a moving visual target in the rhesus monkey. J Neurophysiol. 2011 Feb;105:883-95.

37. Nowak LG, Bullier J. The timing of information transfer in the visual system. In: Cerebral Cortex: Extrastriate Cortex in Primates (Rockland K, Kaas JH, Peters A, eds), pp 205-41 New York: Plenum. 1997.

38. Schall JD, Thompson KG. Neural selection and control of visually guided eye movements. Annu Rev Neurosci. 1999;22:241-59.

39. Schiller PH. The role of the monkey superior colliculus in eye movement and vision. Invest Ophthalmol. 1972 Jun;11:451-60.

40. von Helmholtz H. Handbuch der physiologischen Optik. Liepzig: Voss, 1867.

41. Wang X, Zhang M, Cohen IS, Goldberg ME. The proprioceptive representation of eye position in monkey primary somatosensory cortex. Nat Neurosci. 2007 May;10:640-46.

42. Shanidze N, Kim AH, Raphael Y, King WM. Eye-head coordination in the guinea pig I. Responses to passive whole-body rotations. Exp Brain Res. 2010 Sep;205:395-404.

43. Shanidze N, Kim AH, Loewenstein S, Raphael Y, King WM. Eye-head coordination in the guinea pig II. Responses to self-generated (voluntary) head movements. Exp Brain Res. 2010 Sep;205:445-54.

44. Ganssle JG. The Art of Programming Embedded Systems. San Diego: Academic Press, 1992.

45. Schlag J, Schlag-Rey M. Colliding saccades may reveal the secret of their marching orders. Trends Neurosci. 1990 Oct;13:410-15.

46. Nienborg H, Bridge H, Parker AJ, Cumming BG. Neuronal computation of disparity in V1 limits temporal resolution for detecting disparity modulation. J Neurosci. 2005 Nov;25:10207-19.

47. Wollman DE, Palmer LA. Phase locking of neuronal responses to the vertical refresh of computer display monitors in cat lateral geniculate nucleus and striate cortex. J Neurosci Methods. 1995 Aug;60:107-13.

    Licensee to OAPL (UK) 2014. Creative Commons Attribution License (CC-BY)
    Keywords
    Peer-reviewers