Fifth International School On Mind, Brain And Education

2010, August 1-6

Hitachi. JAPAN

---

Shedding light on hidden yourself; Optical-Topography decoding of higher cognitive functions
Recent progresses in functional neuroimaging, combined with modern machine learning and statistical algorithms, have made it possible to reconstruct, or “decode,” what sensory stimuli or motor outputs a pattern of neural activities represents. This was first demonstrated in the primary visual and sensorimotor areas, and subsequently in cognitive functions such as attention, decision making and emotional states. However, common functional modalities such as fMRI and EEG provide only partial perspectives of such neural processes; fMRI has superior spatial resolution for locating a neural locus yet its temporal resolution is unsatisfactorily limited. EEG, contrastingly, has high temporal resolution for monitoring temporal dynamics of neural processes yet spatial localization of current sources is notoriously difficult. In addition, fMRI imposes severe physical constraints, prohibiting casual uses in real-life situations. We argue that optical topography (OT) can offer an ideal tradeoff for its relatively fast temporal dynamics that detects sub-second hemodynamics and for its reasonably accurate spatial resolution of centimeter order. OT has been successfully applied to elucidating not only primary sensorimotor functions but also higher cognitive functions. In addition, OT is comparatively inexpensive and has fewer physical constraints. These preferable features of OT make it a candidate for the next-generation brain decoding methods.
We propose a general framework of OT-based functional decoding of hidden neural information that goes unnoticed, with three pilot studies: (1) visual awareness of change in the visual field, (2) maintenance of spatial working memory, and (3) degree of skill acquisition and its consolidation. Conventional decoding studies have mainly focused on a precise reconstruction of sensory representations (e.g., a visual image on the retina) or motor outputs (e.g., hand movement trajectories). We extend these studies to decoding “hidden” neural representations that are subconscious and unknowable even to ourselves. We illustrate our main idea by focusing on the first example of visual change awareness, i.e., to decode, from OT signals, whether a participant notices visual change or not [1]. Recent psychophysical studies have shown that our visual system surprisingly fails to notice large and obvious changes in a visual scene if those changes are masked with attentional distractors, a phenomenon known as change blindness. In such situations, our inner representation of a visual scene can differ from the actual, physical one. We demonstrate that OT signals successfully classified whether or not a participant noticed a change or not with the probability of more than 80%. We found two distinct temporal types in the classification probability: a postdictive one that reflected visual awareness of a preceding change, and a predictive one that anticipated visual awareness of a subsequent change. To achieve a high probability of classification, we found that OT signals from multiple cortical sites were needed, suggesting the global and distributed network of visual attention. This encouraging result will indicate a future brain-machine interface that explores our hidden innerselves.