https://orcid.org/0000-0002-1418-6489
Figures
Abstract
Reluctance to make eye contact during natural interactions is a central diagnostic criterion for autism spectrum disorder (ASD). However, the underlying neural correlates for eye contacts in ASD are unknown, and diagnostic biomarkers are active areas of investigation. Here, neuroimaging, eye-tracking, and pupillometry data were acquired simultaneously using two-person functional near-infrared spectroscopy (fNIRS) during live “in-person” eye-to-eye contact and eye-gaze at a video face for typically-developed (TD) and participants with ASD to identify the neural correlates of live eye-to-eye contact in both groups. Comparisons between ASD and TD showed decreased right dorsal-parietal activity and increased right ventral temporal-parietal activity for ASD during live eye-to-eye contact (p≤0.05, FDR-corrected) and reduced cross-brain coherence consistent with atypical neural systems for live eye contact. Hypoactivity of right dorsal-parietal regions during eye contact in ASD was further associated with gold standard measures of social performance by the correlation of neural responses and individual measures of: ADOS-2, Autism Diagnostic Observation Schedule, 2nd Edition (r = -0.76, -0.92 and -0.77); and SRS-2, Social Responsiveness Scale, Second Edition (r = -0.58). The findings indicate that as categorized social ability decreases, neural responses to real eye-contact in the right dorsal parietal region also decrease consistent with a neural correlate for social characteristics in ASD.
Citation: Hirsch J, Zhang X, Noah JA, Dravida S, Naples A, Tiede M, et al. (2022) Neural correlates of eye contact and social function in autism spectrum disorder. PLoS ONE 17(11): e0265798. https://doi.org/10.1371/journal.pone.0265798
Editor: Vanessa Carels, PLoS ONE, UNITED STATES
Received: March 7, 2022; Accepted: October 6, 2022; Published: November 9, 2022
Copyright: © 2022 Hirsch et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data from this publication have been submitted to DRYAD (https://datadryad.org/) and are available at https://doi.org/10.5061/dryad.s4mw6m994.
Funding: National Institute of Mental Health, R01MH111629, Dr. Joy Hirsch National Institute of Mental Health, R01MH107513, Dr. Joy Hirsch National Institute of Mental Health, R01MH119430, Dr. Joy Hirsch National Institute of Mental Health, U19MH108206, James C. McPartland National Institute of Mental Health, R01 MH107426, James C. McPartland National Institute of Mental Health, R01MH100173, James C. McPartland National Institutes of Health, T32GM007205, Swethasri Dravida.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Autism spectrum disorder (ASD) is a complex neurodevelopmental condition including distinct behavioral, communicative, and social responses such as reluctance to make eye contact during natural social interactions [1]. The underlying neurobiology of ASD is typically investigated by single non-interacting participants and, therefore, the biological underpinnings of observed behavioral differences during live and interactive social processes are not well understood. The single participant focus of conventional neuroimaging is a contributing factor to this knowledge gap, as the neural systems that underlie social differences in ASD cannot be directly investigated in passive stimulus-response paradigms. Nonetheless, single participant stimulation paradigms have contributed the foundation for the basic neuroscience of social ability in ASD.
For example, in the case of visual sensing and low-level visual processes, variations in patterns of eye-gaze within the population with ASD are well documented [2–10]. Reduced responses to emotional cues conveyed by simulated facial dynamics [11] and reduced production of facial expressions that signal emotional content [12–15] have also been reported. Reduced occipital pole responses to pictured eyes for participants with ASD observed by fMRI suggest atypical early visual processing of faces [16]. Eye gaze at pictured eyes with emotional content has been associated with abnormally high activation in subcortical systems including superior colliculus, pulvinar nucleus of the thalamus, and the amygdala. Elevated amygdala responses to neutral faces have also been associated with eye gaze suggestive of increased arousal for face stimuli [17]. Electroencephalography (EEG) findings in ASD also show increased latency of event-related potentials (ERPs) to pictured eye stimuli relative to typically developed (TD) participants [18, 19]. This latency has been related to specific attention directed at the eyes of a pictured face in individuals with ASD [20]. Atypical and reduced responses to simulated faces and robots have also been shown using fNIRS in support of the hypothesis of alternative neural pathways for face processing [21].
Temporal cortex is activated specifically during viewing of eye and mouth movements in the TD population [20, 22–24]. In contrast, in ASD, hypoactivation of superior temporal sulcus and face-related dorsal areas consisting of the somatosensory and premotor cortex associated with face processing has previously been reported [25]. These findings are consistent with atypical brain activation patterns during face-to-face joint attention in adults with ASD within the superior temporal sulcus and dorsal medial prefrontal cortex [26]. Hypoactive high-level cognitive and language systems have also been reported [27, 28]. These prior findings of atypical early and higher-order perceptual, cognitive, and language processing are consistent with the overarching hypothesis of impacted social-communicative systems related to face and eye-processing in ASD [29].
Dynamic and reciprocal face interactions are primary sources of social information. In natural real-life situations, humans typically perceive complex facial information conveyed by facial expressions within very brief exposures including subliminal presentations of fearful faces [30]. However, autistic adolescents and adults have reported struggling with flexibly and strategically extracting information from faces during face-to-face interactions [31]. The importance of investigating real-life facial interactions in ASD has been recognized by recent calls for “second-person neuroscience” [32–34]. A theoretical context for this notion in clinical applications has been proposed as “second person neuropsychiatry” [35] aimed at the development of quantitative assessments of dyadic social interactions referred to “interaction-based phenotyping” [36]. A notable example is the dialectical misattunement hypothesis which views psychopathology as a matter of both single brains and also social interactions within dyads [37]. As such, the model emphasizes that psychiatric conditions including ASD are multi-faceted and encompass not only static conditions but also biological, cognitive, behavioral, and socio-cultural factors. However, the inadequacy of neuroimaging techniques for acquiring dyadic information on dynamic face processing during real social interactions has challenged advances in the development of these methods and in understanding the underlying neurobiology of these processes [38, 39] and their variations in ASD.
In response to this knowledge gap, recent developments in functional near-infrared spectroscopy (fNIRS) have applied hyperscanning (simultaneous brain scanning of two individuals during live interactions) to pave the way for much-needed studies of live interactions between individuals and investigations of neural substrates associated with atypical social interactions. The dual brain paradigm provides the infrastructure to quantifiably investigate dyadic behaviors. One potential advantage of two-person paradigms is the application of neural coupling as a measure of dyadic interaction hypothesized to represent sensory and motor processes during cross-brain information exchanges [32]. However, neural coupling may also provide a measure of social “attunement” as envisioned by the dialectical misattunement hypothesis for autism [37]. If so, it would predict reduced neural coupling during interaction for the ASD group. Technical advances in fNIRS [40, 41] and the immediate need to understand the biological components of live and interactive human social behaviors have supported the emergence of this neuroimaging technology and supporting computational approaches to investigate dynamic face-to-face and eye contact behaviors in ASD.
The current investigation aims to identify neural systems responsive to in vivo eye contact in both ASD and TD, and to examine their relationship to social performance. The overarching hypothesis is that interactive face processing, with eye contact being a central component, engages high-level demands for rapid interpretation of subtle eye movements and facial cues that convey social meaning. These cues are not included in non-interactive and conventional simulated face stimuli. Thus, it is expected that neural processing of interactive faces in ASD and TD groups will include social systems [42]; interactive face processing systems [43–45]; and motion-sensitive systems [46] hypothesized to be differentiated between TD and ASD groups. Here we test the specific hypothesis that individual differences in social function as measured by gold standard clinical metrics in ASD are associated with neural responses during live eye-to-eye contact.
TD and adults with ASD were compared during real gaze at the eyes of a same-sex lab partner to gaze at a comparable dynamic face video (instructions were to gaze at the eyes under both conditions). In the real-person interaction condition (Real Eye), partners viewed each other’s faces directly while sitting across a table from one other. Findings were compared with a condition in which participants and their lab partners both viewed size-matched faces displayed on a video monitor (Video Eye). The contrast between these two conditions and direct comparisons between ASD and TD groups tested the hypothesis that neural processes responsive to real eye-to-eye contact are altered in ASD relative to TD. Eye tracking and pupillometry during these conditions were used to test the related hypothesis that gaze characteristics (visual sensing) and an autonomic indicator of arousal (pupil size variation) also varied during face processing between TD and ASD. The social performance for all participants with ASD was assessed by clinical interview, including administration of the Autism Diagnostic Observation Schedule (2nd Edition, ADOS-2) [47]. Self-report measures were given to both participants with ASD and TD participants, including the Social Responsiveness Scale (Second Edition, SRS-2) [48]. A goal of this investigation was to determine how these social metrics of behavioral function related to eye contact were linked to the underlying neurophysiology.
Methods and materials
Participants
Participants included 17 healthy adults with Autism Spectrum Disorder (ASD) (3 female; mean age 25±4.9 years; 12 right-handed, 3 left-handed, and 2 ambidextrous [49]) whose diagnoses were verified by gold standard, research-reliable clinician assessments, including the Autism Diagnostic Observation Schedule, 2nd Edition (ADOS-2 [47]) (S1 Table), and expert clinical judgment using DSM-5 criteria [1]; and 19 healthy, typically-developed (TD) adults (mean age 26±5.8 years; 18 right-handed and 1 ambidextrous) (S2 Table). Determination of sample size sufficient for a conventional power of 0.80 was based on contrasts (Real face > Video Face) reported in a previous study using similar two-person techniques with a lab partner [45]. Using the pwr package of R statistical computing software [50], a significance level of p < 0.05 is achieved with 16 participant/lab partner dyads. Sample sizes 17 ASD dyads and 19 TD dyads ensure adequate effect sizes for these paired experiments for each group. Although sufficient for the planned and reported analyses, a larger sample size would ordinarily be preferred. However, COVID-related circumstances have prevented further acquisitions.
Participants were recruited from ongoing research in the McPartland Lab, the Yale Developmental Disabilities Clinic, and the broader community through flyers and social media announcements. Inclusion criteria included age 18–45 years, IQ≥70, and English speaking. Exclusion criteria included diagnosis of bipolar disorder, personality disorder, or schizophrenia spectrum disorder; anti-epileptic, barbiturate, or benzodiazepine medication use; history of seizures, brain damage, or recent serious concussion; alcohol use within 24 hours; recreational drug use within 48 hours; chronic drug abuse; medication changes within two weeks; sensory impairment or tic disorder that would interfere with fNIRS recording; history of electroconvulsive therapy; or genetic or medical condition etiologically related to ASD. Additional exclusionary criteria for TD participants included self-report of any psychiatric diagnosis or learning/intellectual disability; psychotropic medication; or a first-degree relative with ASD. All participants provided written and verbal informed consent in accordance with guidelines and regulations approved by the Yale University Human Investigation Committee (HIC #1512016895) and were paid for their participation. Assessment of the participants with ASD capacity to give informed consent was provided by a consensus of trained professional staff who monitored the process and confirmed verbal and non-verbal responses. In order to assure that participants were comfortable during the experimental procedure, participants with ASD were accompanied at all times by a member of the clinical team, who continuously evaluated their sustained consent to participate.
All participants were characterized by gender, age, full-scale IQ (FSIQ-4 as estimated by the Wechsler Abbreviated Scale of Intelligence, 2nd Edition (WASI-II; [51]), and self-reported clinical characteristics on several questionnaires, including the Autism-Spectrum Quotient (AQ; [52]); Broad Autism Phenotype Questionnaire (BAPQ; [53]); Social Responsiveness Scale, Second Edition (SRS-2; [48]); Beck Anxiety Inventory (BAI; [54]); State-Trait Anxiety Inventory (STAI; [55]); and the Liebowitz Social Anxiety Scale (LSAS; [56]). See S3 and S4 Tables for detailed demographic and statistical comparisons between the two groups. Group comparisons of clinical assessments indicated expected differences on the AQ (p≤0.01); BAPQ (p≤0.01); SRS-2 (p≤0.01); and BAI scales (p≤0.05), and failed to provide evidence for differences on the WASI-II, STAI, and LSAS between the groups. Assessment and diagnostic tests were performed in clinical facilities at the Yale Child Study Center.
Participants were escorted from the clinical environment to the research environment for fNIRS / eye tracking experiments. An investigator was present during the data acquisition and monitored signs of discomfort during the experiment. All participants were paired with a same-gender TD lab partner. One male (27–28 years old throughout data collection) and one female (22–23 years old throughout data collection) served as lab partners throughout the entire study. Lab partners were not informed of the participant’s group membership before the experiment.
Experimental procedures and paradigm
Dyads (participant and a gender-matched lab partner) were seated ~140 cm across a table from each other and were fitted with an extended head-coverage fNIRS cap. Each participant was instructed to look straight ahead either at their partner or at a monitor with a video face adjusted in size to subtend the same visual angles as the real face (Fig 1A and 1B). In the live (Real Eye) task, dyads were instructed to gaze at each other’s eyes during cued 3-second epochs (1A), and in the video (Video Eye) task, dyads were instructed to gaze at the eyes of the face as it appeared in the dynamic video (1B). The illustrative red box enclosing the eyes of the participants in Fig 1 subtended 3.3x1.5° of visual angle and defined the location of the “eye box,” a region designated as the eye contact zone for each participant. In both tasks, dyads alternated their gaze between the eyes of their (real or video) partner and two small light-emitting diodes (LEDs) located 10° to the left and 10° to the right of their partner’s face (Fig 1C and 1D). The video was a recorded version of a same-gender participant performing the same task while wearing the same optode cap as live participants.
A. Gaze at partner’s eyes: Real Eye condition. Partners viewed each other at an eye-to-eye distance of 140 cm. The eye regions subtended by both the real eyes and the video eyes were 3.3 × 1.5 degrees of visual angle (red boxes). Small green LED indicator lights located to either side of their partner indicated rest and diverted gaze targets. B. Gaze at eyes in video: Video Eye condition. Two 24-inch 16x9 monitors were placed between the participants and a size-calibrated, pre-recorded dynamic video of a face was presented in the same field-of-view as the live interaction. C. Diagram of the Real Eye condition, with participant and lab partner sitting 140 cm apart from each other and LED indicator lights placed 10 degrees to the left and right of the Eye. D. Diagram of the Video Eye condition, with monitors arranged between partners. The face and LED sizes and positions were calibrated to subtend the same visual angles in both conditions.
The order of runs was randomly sequenced between viewing a real partner directly or viewing the visual-angle corrected video partner on a 24-inch 16x9 computer monitor placed back-to-back between participants, including a partition to assure that dyads could not see their real partner during video conditions. The face and distance of the video stimuli were calibrated to subtend identical degrees of visual angle in the field of view of the participants and the timing and range of motion of eye movements between partners were the same in both tasks. The time-series and experimental details were similar to prior studies [43, 45] and are included here to provide a self-contained report.
At the start of each task, an auditory cue prompted participants to gaze at the eyes of their real or recorded partner. Subsequent auditory tones alternatingly cued eye gaze between eyes or LED according to the protocol time series. The 15-second active task period alternated with a 15 s rest/baseline period. The task period consisted of three 6-second cycles in which gaze alternated on the partner for 3 s and then on a lighted LED to either the right or left (alternating) of the participant for 3 s for each of three events. The time series was performed in the same way for all runs. The order of runs was counterbalanced across pairs of participants. During the 15 s rest/baseline period, participants focused on the lighted LED, as in the case of the 3 s periods that separated the eye contact and gaze events. The 15 s activity epoch with alternating eye contact events was processed as a single block.
The experimental paradigm (Fig 2A) employed a classic hemodynamic time series with 15 s of task alternating with 15 s of rest. Run length was 3 m and included six task-rest cycles. Due to the social discomfort associated with prolonged mutual gaze at another’s eyes, the task epochs were subdivided into events (epochs) that alternated between three 3-second “eye-on” and 3-second “eye-off” cycles (see Fig 2A). During the “eye-on” epoch, dyads were instructed to gaze at the eyes of their (real or video) partner, making eye contact as often as possible in natural intervals. An auditory tone signaled the transition between eye-on and eye-off events indicating when participants were instructed to divert their gaze to the LED targets 10° to the right or left.
A. Time course. The duration of the run was three minutes and each run was repeated twice for both the Real Eye and Video Eye conditions. Each run included six alternating 15-second task and rest periods. In task periods (blue bars), participants alternated their gaze in three-second epochs between the eyes and the left or right lighted LED (See Fig 1C and 1D). During the 15-second rest period, participants looked only at the lighted LED. The task is similar to those used in previous experiments (Hirsch et al., 2017; Noah et al., 2020). B. Eye tracking traces of eye-to-eye contact. Red traces represent eye movements from a participant with ASD; blue traces represent the eye movements of a lab partner. The eye tracking data acquired on the Tobii system provides a frame-by-frame (8 ms) binary value that indicates whether or not eye gaze was directed within the eye-box of the partner. The blue dashed line (top) represents the duration of eye gaze (number of frames) that the lab partner’s gaze was within the eye-box of the participant. Similarly, the red dashed line (bottom) represents the duration of gaze (number of frames) that the participant’s eye gaze was in the eye-box of the lab partner. The green dashed line (middle) represents the length of time (number of frames) that the eyes of both partners were simultaneously focused within each other’s eye-boxes for a minimum of 83 ms. This is taken as a measure of eye-to-eye contact between the participant and the lab partner. C, D. Gaze performance, Real Eye condition. Eye contact epochs (3 s) are indicated as E; Diverted gaze epochs (3 s) are indicated as D; and Rest periods (15 s) are indicated as R. The color bar (top) indicates the percent of time eye gaze was within the eye box of the partner. C. Example eye tracking report for one TD participant and lab partner pair. D. Example eye tracking report for one participant with ASD and the lab partner pair. Similar computations were performed for all participants when eye tracking data were acquired.
Eye tracking.
Two Tobii Pro x3-120 eye trackers (Tobii Pro, Stockholm, Sweden), one per participant, were used to acquire simultaneous eye tracking data at a sampling rate of 120 Hz. Eye trackers were mounted on the table facing each participant. Prior to the start of the experiment, a three-point calibration method was used to calibrate the eye tracker on each participant. The partner was instructed to stay still and look straight ahead while the participant was told to look first at the partner’s right eye, then left eye, then the tip of the chin. The same calibration procedure for video interactions was performed before recording on a still image presented on the monitor 70 cm in front of the participants. Similar live calibration procedures have been used successfully in prior investigations of in-person social attention [57, 58]. As instructed for the eye movement task, participants alternated their gaze between ≈0° and 10° of deflection. Participants fixated on the eyes of the video (Video Eye condition) or the eyes of the lab partner (Real Eye condition) ±10° deflections to either the left or right. The eye contact portions of the task were 3 s in length, with six per trial, for 18 s of expected eye contact over the trial duration (Fig 2B).
Eye tracking provided a measure of compliance with task instructions to fixate on the eye as illustrated in Fig 2B for a lab partner (blue trace) and a participant with ASD (red trace). The x-axis represents the run time series (180 s), and the y-axis represents the gaze angle, where 0° represents eye-to-eye contact and ±10° indicate left and right deflections, respectively. The moments of dyadic eye contact (gaze is within the eye box of their partner) are indicated by the green line. The time series of Fig 2A and 2B are synchronized for illustrative purposes. The blue and red dashed lines above and below the eye position trace indicate the times of gaze locations that are within the eye box of the partner for the lab partner and the participant with ASD respectively. An “eye box hit” is defined when the gazes of both partners are within the designated eye box of the other for a minimum of 83 ms, i.e., 10 frames [59]. The green lines in the figure indicate these 10-frame time points where the gaze of both partners was in the eye box of the other. The eye contact performance is illustrated for a typical participant in Fig 2C and a participant with ASD in Fig 2D by the color of the event bar where the percentage of time in the eye box of the lab partner is represented by the color bar for the entire run time (180 s). Examples of average gaze positions are shown for 5 typical participants in Fig 3A to illustrate the gaze and eye box confirmation.
A. Examples of average participant eye gaze positions when viewing the face of the lab partner. The red box illustrates the target “eye box” and the color gradient from red to green indicates percent of “target hits” in the eye box for an entire run. B. fNIRS channel layout. Right and left hemispheres of a single rendered brain illustrate median locations (blue dots) for 58 channels per participant. Montreal Neurological Institute (MNI) coordinates were determined for each channel by digitizing emitter and detector locations in relation to anterior, posterior, dorsal, and lateral fiduciary markers based on the standard 10–20 system.
Functional NIRS signal acquisition and channel localization.
Functional NIRS signal acquisition, optode localization, and signal processing, including global mean removal, were similar to methods described previously [43, 60–65] and are briefly summarized below. Hemodynamic signals were acquired using 3 wavelengths of light, and an 80-fiber multichannel, continuous-wave fNIRS system (LABNIRS, Shimadzu Corp., Kyoto, Japan). Each participant was fitted with an optode cap with predefined channel distances. Three sizes of caps were used based on the circumference of the heads of participants (60 cm, 56.5 cm, or 54.5 cm). Optode distances of 3 cm were designed for the 60 cm cap but were scaled equally to smaller caps. A lighted fiber-optic probe (Daiso, Hiroshima, Japan) was used to remove all hair from the optode channel before optode placement.
Optodes consisting of 40 emitters and 40 detectors were arranged in a custom matrix, providing a total of 54 acquisition channels per participant. The specific layout with the coverage of the optode channels is shown in Fig 3B. For consistency, placement of the most anterior channel of the optode holder cap was centered 1 cm above nasion. To assure acceptable signal-to-noise ratios, signal quality was measured for each channel prior to recording, and adjustments were made for each channel until all recording optodes were calibrated and able to sense known quantities of light from each laser wavelength [61, 66, 67]. Anatomical locations of optodes in relation to standard head landmarks were determined for each participant using a Patriot 3D Digitizer (Polhemus, Colchester, VT) [68–72]. Montreal Neurological Institute (MNI) coordinates [73] for each channel were obtained using NIRS-SPM software [74] with WFU PickAtlas [75, 76].
Analysis of eye-to-eye contact, dwell time, and pupillary responses.
Eye tracking information including pupil size was exported from the Tobii system to a data processing pipeline and custom MATLAB scripts in where acquired data were used to calculate the mutual eye contact events, accuracy, latency to targets, and pupil diameters. Eye-tracking data were not usable on 5 out of 17 participants with ASD and 4 out of 19 TD participants due to either calibration or equipment problems (right columns of S1 and S2 Tables summarize the eye tracking acquisitions). Tobii Pro Lab software (Tobii Pro, Stockholm, Sweden) was used to create areas of interest for subsequent eye tracking analyses run in MATLAB (Mathworks, Natick, MA). The eye box was identified manually for each run and each participant for both live and video sequences. For the measures of gaze duration and variability, the horizontal components of gaze trajectories were gated by the eye-to-eye portions of each trial, retaining only samples that were within the eye box range.
This analysis used zero angle (eye contact) intervals to characterize participant eye contact behavior. The eye tracking source was the horizontal component of post-processed trajectories converted to units of arc length (tenths of a degree). There were 1350 observations of 27 participants (15 TD, 12 with ASD). To avoid possible inclusion of the large movements into and out of the valid range, the first and last 200 ms of each 3 s eye contact interval were excluded. Three measures were obtained from each interval: Dwell Time, the number of valid retained samples per interval normalized by sampling rate (seconds); Gaze Variability, the standard deviation of the samples centered over each interval, normalized by the number of retained samples (Fig 4A and 4B); and pupil diameter (mm) Fig 4C and 4D. In the case of the gaze position data (Fig 4A and 4B) linear mixed-effect models were used to assess the fixed effects of group (TD, ASD) and condition (Video Eye, Real Eye), with random intercepts by participant. Pupil sizes for left and right eyes (Fig 4C and 4D) were sampled at 40 Hz using the Tobii eye tracking system. All analyses used average pupil sizes across both eyes. To match the temporal resolution of the gaze position data, the pupil diameter data were interpolated to a sample rate of 120 Hz.
A, B. Marginal means plots based on linear mixed effects models for each measure by participant group (Blue: Participants with Autism Spectrum Disorder (ASD); Red: Typically-developed (TD) participants). A. Dwell time duration of eye contact on either the eyes in the video (Video Eye condition) or eyes of the lab partner (Real Eye condition). B. Standard deviation of horizontal gaze trajectory normalized by duration of contact. Error bars show SEM. ***p≤0.001, *p≤0.05. C, D. Pupil diameter variations observed in participants with ASD and TD participants during Real Eye (C) and Video Eye (D) gaze conditions (Blue: Participants with ASD; Red: TD participants). Black lines indicate points at which pupil diameter differences between groups were significant at p≤0.05.
fNIRS signal processing.
Raw optical density variations were acquired at three wavelengths of light (780 nm, 805 nm, 830 nm), which were translated into relative chromophore concentrations using a Beer-Lambert equation [77–79]. Signals were recorded at 30 Hz. Baseline drift was removed using wavelet detrending provided in NIRS-SPM [74]. In accordance with recommendations for best practices using fNIRS data [80], global components attributable to blood pressure and other systemic effects [81] were removed using a principal component analysis (PCA) spatial global mean filter [60, 62, 82] prior to general linear model (GLM) analysis. All analyses are reported using the combined OxyHb and deOxyHb signals (Hb Diff). The combined signal averages are taken as the input to the second level (group) analysis [83]. Comparisons between conditions were based on GLM procedures using the NIRS-SPM software package. Event epochs within the time series were convolved with the hemodynamic response function provided from SPM8 [84] and were fit to the signals, providing individual “beta values” for each participant across conditions. Group results based on these beta values were rendered on a standard MNI brain template (TD-ICBM152 T1 MRI template [73] in SP8 using NIRS-SPM software [74] with WFU PickAtlas [75, 76].
Coherence signal processing.
Methods for calculating neural coherence (synchrony) are the same as those used in previous experiments [45, 64, 85]. Briefly, cross-brain coherence between homologous pairs of brain regions was evaluated using wavelet analysis [86, 87] with the combined Oxy- and deOxyHb signals in the MATLAB Wavelet Toolbox. The wavelet kernel was a complex Gaussian provided by MATLAB. The number of octaves was four, and the range of frequencies was 0.4 to 0.025 Hz. The number of voices per octave was also four; therefore, 16 scales were used for which the wavelength difference was 2.5 seconds. Individual channels were grouped into regions based on shared anatomy, which served to optimize signal-to-noise ratios. Grouping was achieved by identification of 14 bilateral ROIs from the acquired channels, including: 1) angular gyrus (BA39); 2) dorsolateral prefrontal cortex (BA9); 3) dorsolateral prefrontal cortex (BA46); 4) pars triangularis, BA45; 5) supramarginal gyrus (BA40); 6) middle temporal gyrus (BA21); 7) superior temporal gyrus (BA22); 8) somatosensory cortex (BA1, 2, and 3); 9) somatosensory association cortex (BA7); 10) pre-motor and supplementary motor cortex (BA6); 11) subcentral area (BA43); 12) inferior frontal gyrus (BA47); 13) visual cortex (Area V3, BA19); and 14) frontal eye fields (BA8). Signals acquired from predefined anatomical regions were decomposed into a range of temporal frequencies that were correlated across two brains for each dyad. Here, we apply the decomposed “residual signal” to investigate effects other than the main task-induced effect. This technique effectively removes the task covariate as is conventional for PsychoPhysiological Interaction (PPI) analysis [88]. For example, cross-brain coherence of multiple signal components (wavelets) is thought to provide an indication of dynamic coupling processes rather than task-specific processes, which are coupled by virtue of the coordinated task. Coherence was compared for Real Eye gaze and Video Eye gaze conditions for TD and ASD groups.
Results
Behavioral
Eye-to-eye contact.
Even though eye-to-eye contact is often reduced in individuals with ASD, in this investigation, we asked our participants to look directly at the face of the lab partner and to make eye-to-eye contacts during the cued 3 second periods. The recorded measures of gaze time in the “eye box” did not differ systematically between TD and participants with ASD for either the Real Eye or Video Eye conditions (see S1 and S2 Figs), confirming compliance with this task. A t-test of median eye box dwell time percentages failed to provide evidence for differences between groups (see S5 Table). This approach supports the assumption that both TD and participants with ASD performed the same task, i.e., eye-to-eye contact during the 3s epochs. Gaze dwell time assessed using a linear mixed effects model with fixed effects of Group and Condition and random intercepts by participant showed no group difference or interaction, but dwell time for both TD and ASD groups was significantly longer in the Real Eye condition (t = 10.88, p≤0.001). See Fig 4A. However, gaze variability (assessed as the standard deviation of the horizontal component of eye trajectory during the eye contact intervals normalized by their duration) was greater for the ASD than the TD group for both conditions, see Fig 4B, consistent with altered visual sensing mechanisms in eye-movement patterns in ASD while viewing the face and eye stimuli in either condition (t = 2.08, p≤0.05). Event-triggered averages of pupil diameter measurements were compared for the two conditions, real face-to-face (Fig 4C) and video face/eye gaze (Fig 4D). While both groups initially showed pupil dilation for gaze at real eyes, overall dilation in the ASD group was greater than in the TD group (p<0.05). No evidence for differences was observed between groups during gaze at the eyes in a video face suggesting increased autonomic responses to real faces, but not video faces, in ASD [89].
Neural findings
Direct comparison of TD and ASD findings for real eye-to-eye contact.
A test of the alternative neural pathways hypothesis for live and interactive face processing in ASD is shown in Fig 5 with a direct comparison of the TD and ASD neural responses during live eye-to-eye contact. Red clusters indicate increased neural activity for the TD group and blue clusters indicate increased neural activity for the ASD (p≤0.05, FDR-corrected). Neural activity in the Real Eye condition is increased in the dorsal parietal regions for TD> ASD and in the temporal and ventral frontal regions for ASD>TD. See caption and tables for specific anatomical details.
TD participants (red clusters) show comparatively greater activation in dorsal somatosensory cortex (SSC); supramarginal gyrus (SMG); angular gyrus (AG); pre- and supplementary motor cortex (MC); and extrastriate visual cortex (V3), while heightened activity was observed for participants with ASD (blue clusters) in relatively ventral MC; SSC; pars opercularis (Pars O) and pars triangularis (Pars T); posterior middle temporal gyrus (pMTG); superior temporal gyrus (STG); and auditory cortex (AC). See Table 1. Yellow and light blue indicate responses corrected for multiple comparisons using FDR at p≤0.05. GLM analyses are based on the combined OxyHb and deOxyHb signals.
Modulation of neural circuitry by frequency of eye contact events.
A further test of the ventral vs dorsal alternative pathways hypothesis is shown in Fig 6. Neural responses during each 3s eye viewing period were modulated by the number of eye contact events within that period for both TD and ASD groups. The covariance variable of eye-contacts used in the second level (group) analysis was constructed by assigning each subject with the median eye contact time for the 3s periods where the eye of the partner was viewed. For TD participants (Fig 6A) clusters were observed in right dorsal supramarginal gyrus (SMG), somatosensory association cortex (SSAC), and dorsolateral prefrontal cortex (DLPFC); frontal eye fields (FEF); and pre- and supplementary motor cortex (MC) (see Table 2A). In sharp contrast to these TD observations, participants with ASD (Fig 6B) neural responses to eye-to-eye signals modulated by the same measures of eye contact events were observed in the ventral right supramarginal gyrus (SMG); angular gyrus (AG); extrastriate visual (V3) and visual association cortices (V2); as well as the dorsolateral prefrontal cortex (DLPFC). Neural patterns in TD and participants with ASD both demonstrated activity in the DLPFC whereas group response patterns were clearly differentiated in the posterior regions of the brain. In the case of TD participants, dorsal parietal regions were responsive to eye-to-eye contact, while in the participants with ASD, ventral occipital and temporal regions were responsive to eye-to-eye contact. Although with the eye-tracking covariate the contrast analysis is underpowered, these findings serve to illustrate the dorsal and ventral processing streams between TD and ASD groups shown above. In particular, in TD the right dorsal-parietal stream is activated, whereas in ASD, the right ventral occipital-temporal stream is activated.
A. Typically-developed (TD) participants. Activity observed in the right hemisphere: supramarginal gyrus (SMG); somatosensory association cortex (SSAC); dorsolateral prefrontal cortex (DLPFC); frontal eye fields (FEF); and pre- and supplementary motor cortex (MC). See Table 2A. Note: n = 15 rather than 19 (see S2 Table) because eye tracking data could not be acquired on four participants. B. Participants with Autism Spectrum Disorder (ASD). Activity observed in the right hemisphere include SMG; angular gyrus (AG); extrastriate visual cortex (V3); visual association cortex (V2); and DLPFC. See Table 2B. Note: n = 12 rather than 17 (see S1 Table) because eye tracking data could not be acquired on five participants. Yellow indicates signals corrected for multiple comparisons at p≤0.05 using FDR. GLM analyses are based on the combined OxyHb and deOxyHb signals.
B. GLM Contrast comparison: [Real Eye] > [Rest] with eye contact covariate (deOxyHb + OxyHb signals), ASD group.
Neural responses during real eye gaze modulated by symptom severity as measured by ADOS-2: Group effects.
Neural responses (beta values) acquired during eye-gaze were regressed by the individual ADOS-2 scores using the general linear model, GLM. The whole-brain main effect of the eye contact activity modulated by ADOS-2 scores is shown in Fig 7. Blue clusters indicate regions of the brain where neural activity as represented by the individual average was negatively related to the individual ADOS-2 scores. That is, participants with higher ADOS-2 scores and greater symptomatology showed consistently lower live eye contact related neural activity located in the right dorsal parietal areas including angular gyrus (AG), supramarginal gyrus (SMG), somatosensory association cortex (SSAC), and somatosensory cortex (SSC). Similar findings were also observed the right dorsolateral prefrontal cortex (DLPFC) (p≤0.01). See Table 3.
OxyHb and deOxyHb signals are combined. Blue colors indicate a negative relationship between neural responses and ADOS scores indicating that as symptom severity increases, neural responsiveness in these regions decreases. Light blue indicates responses corrected for multiple comparisons using FDR at p≤0.01. Pars O: pars opercularis; Pars T: pars triangularis; DLPFC: dorsolateral prefrontal cortex; SSAC: somatosensory association cortex (or SPL: Superior Parietal Lobule); SMG: supramarginal gyrus; AG: angular gyrus; and V3: Extrastriate Visual Cortex (Area V3). n = 12 instead of 17 because the analysis includes only those participants with usable eye-tracking data. See Table 3.
Neural responses during real eye gaze correlated with symptom severity as measured by ADOS-2: Individual ASD effects.
The individual ADOS-2 scores for each participant (identified by participant number in S1 Table) (x-axis, Fig 8) are plotted against the median beta values, parameter estimations, of the fNIRS signal (y-axis) for the three regions of interest identified by the group effects above: A posterior dorsal stream cluster consisting of the visual cortex and angular gyrus; somatosensory association cortex and supramarginal gyrus; and an anterior cluster in the dorsolateral prefrontal cortex and sections of the inferior frontal gyrus (pars opercularis and triangularis). The best fit lines illustrate the negative relationships and the correlation coefficients represent the strengths of the associations.
The median hemodynamic signals (beta values, y-axis) within the responsive brain regions (Fig 7 and Table 3) and ADOS-2 scores (x-axis) are shown for each participant with ASD. The main effect of eye-to-eye contact is negatively correlated with fNIRS signals in A. right hemisphere visual association cortex (Area V3) and angular gyrus (r = -0.76) as well as B. somatosensory association cortex (also referred to as the superior parietal lobule, SPL) and supramarginal gyrus (r = -0.92); however, due to variations in optode coverage over the SPL (because of differences in head size and the superior-medial location of the target area), the number of participants with sufficient data for this regional analysis is small (n = 6). A negative correlation between individual median hemodynamic signals and ADOS-2 scores for [Real Eye > Rest] was also found in C. right dorsolateral prefrontal cortex, pars triangularis, and pars opercularis (r = -0.77).
Neural responses during real eye gaze correlated with symptom severity as measured by SRS-2: Individual ASD and TD effects.
To further evaluate the relationship between social symptomatology and live-face eye-gaze for participants with ASD and TD participants we combine the SRS-2 scores for both groups based on the assumption that ASD traits are also present in the general population. Consistent with the findings based on the ADOS-2 scores above, a negative relationship (blue cluster in Fig 9) was observed in regions located in the right dorsal stream including somatosensory cortex (SSC), somatosensory association cortex (SSAC), and supramarginal gyrus (SMG). Participants with higher SRS-2 scores indicating higher levels of symptomatology showed reduced neural activity during eye contact in the right somatosensory cortex (SSC), somatosensory association cortex (SSAC), and supramarginal gyrus (SMG) (p≤0.01). Table 4 provides the peak MNI coordinates, cluster t-values, anatomical regions within the cluster, Brodmann’s Area (BA), probability of inclusion in the cluster, and relative size of the active area (n of voxels).
OxyHb and deOxyHb signals are combined. Blue colors indicate a negative relationship between neural responses and SRS-2 scores, which suggests that increased symptom severity is associated with reduced regional neural responsiveness (See Table 4). Light blue indicates responses corrected for multiple comparisons using FDR at p≤0.01. SSC: somatosensory cortex, SSAC: somatosensory association cortex, and SMG: supramarginal gyrus.
SRS-2 scores and neural responses: Individual differences.
The individual SRS-2 scores for each participant (identified by a number that corresponds to the participant number in S1 and S2 Tables) are plotted against the individual median beta values of the fNIRS signal (Fig 10). Red numbers represent TD participants and blue numbers represent participants with ASD. The interspersal of the individual scores between participants with ASD and TD participants is consistent with the assumption that social responsiveness traits vary within the general population as well as within ASD. The best fit line illustrates the negative relationship (r = -0.58). Missing data points are due to incomplete neural data because of sparse optode coverage of the regions of interest for some participants.
The median hemodynamic signals (Beta values, y-axis) within the responsive brain region (Fig 9 and Table 4) and SRS-2 scores (x-axis) are shown for each participant. The main effect of eye-to-eye contact is negatively correlated with fNIRS signals in right hemisphere somatosensory cortex, somatosensory association cortex, and dorsal supramarginal gyrus (r = -0.58). Numbers indicate individual participants shown in S1 (participants with ASD) and S2 Tables (TD participants).
Cross-brain coherence findings.
Fig 11 presents the findings of cross-brain coherence between the angular gyri of dyadic partners. Coherence of signals in angular gyrus is predicted in a prior live-face investigation and it is the basis for the region of interest selection [45]. The x-axis represents the temporal periods (wavelengths) between 5 and 40 seconds, and the y-axis represents the cross-brain coherence as a correlation between wavelets. As the figure key indicates, TD Real Eye conditions (red) demonstrate increased cross-brain coherence relative to the TD Video Eye (dark blue), and neither of the ASD conditions (Real Eye or Video Eye) demonstrate increases in cross-brain coherence.
The x-axis shows the wavelet components (s) plotted against coherence, the correlation of the components across participants. The inset key indicates the four conditions of interest: TD Real Eye (red); TD Video Eye (dark blue); ASD Real Eye (pink); and ASD Video Eye (light blue). Solid lines represent the mean coherence and shading indicates the standard error. The signal is the combined OxyHb and deOxyHb, and the total number of dyads is 36.
Discussion
Summary of findings
Overall, there were four main findings of this investigation that support the alternative neural processes hypothesis for live interactive face gaze in ASD: 1) Neural systems within ventral occipital-temporal regions were engaged in ASD, whereas in TD live interactive face gaze was associated with dorsal parietal and lateral prefrontal regions; 2) Variations in oculomotor and visual sensing were observed in ASD, including increased positional variation in eye fixations and increased pupillary reactions to live faces, suggesting that visual acquisition factors may also contribute to live face processing difficulties in social signaling; 3) A biological basis for social performance associated with ASD is suggested by the association between ADOS-2 and SRS-2 scores and the counter-correlation of neural responses in the right dorsal parietal regions during real eye-to-eye contact; and 4) Cross-brain synchrony observed for TD real eye-to-eye contact was not observed for ASD, suggesting a reduction in the cross-brain transmission of shared face information. This finding is consistent with the prediction of the dialectical misattunement hypothesis of an interactive mismatch between persons [37].
Differences in social performance, including reduced eye contact, are common characteristics of ASD. Although disparities in face processing and oculomotor variations in ASD are well documented, it is not known how these behaviors are linked to the underlying neurophysiology associated with live and natural interactions. In this investigation, we employed a two-person paradigm using functional near-infrared spectroscopy (fNIRS) to acquire neuroimaging responses during live dynamic eye-to-eye contacts with a lab partner that are a close proxy to the behaviors under investigation. Simultaneous eye-tracking and oculomotor responses were also acquired in 17 adult ASD dyads and 19 closely matched TD adult dyads. Social performance was quantified by clinical interview (ADOS-2: Autism Diagnostic Observation Schedule, 2nd Edition) in ASD and self-report (SRS-2: Social Responsiveness Scale, Second Edition) in both ASD and TD to test the hypothesis that individual differences in social function are predicted by neural responses associated with live eye-to-eye contact.
Direct comparisons of neural findings between TD and ASD for the Real Eye condition are consistent with right dorsal parietal activity in the case of TD (TD > ASD) and right ventral occipital-temporal activity in the case of ASD (ASD > TD). Neural findings modulated by real eye-to-eye contact behavior revealed similar findings of increased right dorsal parietal activity for TD and alternatively increased right ventral parietal activity for ASD consistent with the hypothesis of dorsal eye processing systems for TD and ventral eye processing streams for ASD groups. Individual ADOS-2 scores were negatively correlated with individual fNIRS beta values (representing the strength of hemodynamic signals) within clusters in the right dorsal parietal stream including somatosensory cortices, angular gyrus, and supramarginal gyrus. Similarly, SRS-2 scores for the combined ASD and TD groups were negatively correlated with somatosensory cortices and the supramarginal gyrus also located in the right dorsal parietal stream. Since these two clinical measures provide similar information, it is expected that their relationship to underlying neural correlates would be similar. Neural responses in the right dorsolateral prefrontal cortex (DLPFC) during live eye-to-eye contact were also negatively correlated with ADOS-2 scores. These correlations between social function and the neural responses during live eye-to-eye contact are consistent with a model of hypoactive dorsal parietal and dorsal lateral prefrontal cortex in ASD.
Two-person visual sensing in ASD
Pupillometry, a presumed measure of activity associated with the locus coeruleus-norepinephrine system [89] as well as increased arousal and task load [90] revealed increased pupil diameters for ASD during real eye conditions but not the video eye conditions relative to TD (p<0.05), consistent with the interpretation of increased arousal associated with the real face and eyes in ASD. Further, variance of gaze positions was greater for ASD than TD for both real and video faces suggesting oculomotor differences in visual sensing [91] and predicting social signaling variations.
A natural in-person encounter typically involves active visual sensing of dynamic face landmarks [92]. Guidance systems for visual saccades and fixations are thought to actively “seek” relevant visual information such as social cues that are conveyed hierarchically to higher levels of neural processing [91]. The cascade of synchronized oculomotor behaviors, for example, associated with mutual live eye-to-eye contacts does not occur during passive gaze at an inanimate representation of a real person because dynamic behaviors from both partners are required for a mutual eye contact event or a dynamic face-to-face interaction. In this investigation, we include measures of visual sensing, dwell time, and positional variance as well as pupil size to test the hypothesis that in ASD the live two-person condition may be distinguished by oculomotor behaviors in addition to neural processing. Differences in oculomotor functions have been previously reported in ASD, for example, for static and simulated stimuli [93, 94]. Here we test the hypothesis that oculomotor systems are also affected during two-person face-to-face interactions.
The observed increased positional variation in ASD eye movements leads to the speculation that information characterizing an interactive face may not have been sufficiently acquired for participants with ASD. The finding that live face processing in the ASD group increased activity in ventral and lateral occipital and temporal regions, rather than dorsal parietal regions, could be due, in part, to differences in visual sensing. The observed ventral processes are more consistent with non-interactive face functions than with interactive functions. For example, topographical maps associated with regional specializations for coding simulated faces are well-established. The ventral-occipital cortex is highly selective for and sensitive to pictures of faces [95, 96]. Regions within the superior temporal sulcus are involved in detecting dynamic facial movements presented in two-dimensional stimuli [22, 24], and parameterized face processing codes for static faces have been identified by electrophysiology in middle and superior temporal gyri of non-human primates [97].
Live two-person interactions in ASD
Neuroimaging based on functional near-infrared spectroscopy (fNIRS) enables simultaneous acquisition of hemodynamic brain signals from two individuals (hyperscanning) dynamically engaged in natural interactions. Eye tracking acquired simultaneously on both participants during face-to-face engagement enables identification of eye contact events that occur between the partners. The aim to understand the neural mechanisms that underlie social function in ASD has motivated this multi-modal application of fNIRS and eye tracking. In spite of the biological significance of live interpersonal interactions for survival and social well-being, the underlying neural processes of interactive behaviors are relatively novel targets of investigation for natural settings as well as clinical, developmental, and psychiatric applications [32–34, 98]. Fundamental models of dynamic and reciprocal behaviors are under development for multiple sensory and communication systems, clinical applications, and social behaviors [32–34, 99–104].
Current models of face and eye processing in TD and ASD are based primarily on non-interactive paradigms where data are acquired in single-subject situations using conventional stimulus and response models rather than dyadic paradigms that include live social interactions. Understanding neural activity during natural interactions is especially critical in ASD, as the defining social and communicative characteristics of the condition are often attenuated or absent during explicit laboratory tasks [39]. The importance of investigations that include natural and dynamic two-person interactions between individuals is highlighted by a general theoretical framework proposed by the Interactive Brain Hypothesis [105, 106], which suggests that live interactions between individuals engage neural functions not activated during similar tasks performed alone, i.e., without interaction. A rapidly emerging neuroimaging literature and theoretical framework of live and natural interactions compared to single-subject interactions contributes an accumulating body of evidence in support of this hypothesis [26, 32, 34, 38, 64, 104, 107]. The dialectical misattunement hypothesis extends these notions to large scale social and cultural factors that influence social interactions and emphasizes the role of interpersonal coupling during social interaction [37].
This long-standing experimental paucity of two-person interactive experimental paradigms in ASD, in part, reflects the historical limitations of conventional neuroimaging methods. For example, in functional magnetic resonance imaging (fMRI) solitary confinement in the bore of a scanner with minimal tolerance of head movements constrains/contraindicates investigations of natural, two-person interactions. Fortunately, however, these particular limitations are substantially resolved by recent developments of optical neuroimaging, functional near-infrared spectroscopy (fNIRS), a non-invasive spectral absorbance technique that detects changes in blood oxygen levels in both oxyhemoglobin and deoxyhemoglobin using surface-mounted optical sensors [108–111]. Functional NIRS enables simultaneous acquisitions of hemodynamic signals (assumed to be a proxy for neural activity as in fMRI) from naturally interacting dyads and provides simultaneous dyadic measures that contribute to understanding interactive behaviors as opposed to single-subject responses that focus primarily on perceptual and cognitive systems.
Social performance and face processing
This application of two-person neuroimaging technology to investigate the relationship between the neural underpinnings of interactive face and eye contact and social performance in ASD addresses a prominent and understudied question. Individual clinical evaluations of social performance applied as a second level covariate on whole-brain neuroimaging findings acquired during live real person eye-to-eye contacts confirm a negative relationship between test scores and neural signals in brain regions responsive to real eye-to-eye contacts. Participants with higher ADOS-2 scores, reflecting reduced social performance, showed lower neural signals (beta values, an indicator of signal strength and fit to the general linear model) in brain regions previously associated with social activity, interactive face processing, and motion sensitivity. Findings also included the right dorsolateral prefrontal cortex, a region implicated in both ASD and commonly co-occurring conditions, such as major depressive disorder. Further, a similar finding was observed for the SRS-2 when the scores of both TD and participants with ASD were combined for regions within the dorsal stream but not the DLPFC. That is, as individual social ability decreased as indicated by the elevated SRS-2, the neural signal decreased in the right dorsal-stream regions. A similar finding was observed for SRS-2 scores and the relationship to neural signals acquired by fMRI in the fusiform gyrus and the amygdala during static face processing [112]. Interestingly, in this study, the SRS-2 finding included TD as well as participants with ASD suggesting that variations in social responsiveness and the associated reduction in dorsal stream activity are similarly represented in the general population.
Unique features of this study include the live interactive eye-to-eye task as well as the eye-tracking documentation of compliance combined with the individual difference approach to characterize single participants by both measures. All data were acquired during the epochs when participants directed their eye gaze to within the eye-box of the lab partner. Continuous monitoring confirmed high levels of compliance in all cases. That is, when asked to perform the task, participants were able to do it although eye-to-eye contact was not necessarily a comfortable task for them.
These findings of a negative association between right dorsal regions and social performance do not imply a causal role between neural substrates and reduced social function. However, it can be concluded that the dorsal regions found to be related to symptomatology (right hemisphere angular gyrus, supramarginal gyrus, somatosensory association cortex, somatosensory cortex, and dorsolateral prefrontal cortex), are involved in the underlying neural conditions relevant to ASD. Given a well-recognized need for biomarkers for ASD that associate with the clinical phenotype at the individual level, the strong relationships observed between neural activity and both clinician and self-reported social function suggest potential utility in key contexts of use, such as stratification for enrichment of clinical trials [113].
In summary, these findings highlight the right dorsal stream system and interactive face processing as regions and tasks of interest for understanding the underlying neural mechanisms that distinguish participants with ASD and TD participants. The specificity of these findings opens new directions for investigating these brain-to-behavior linkages. For example, these regions have previously been implicated in motion sensitivity [46] and raise the interesting hypothesis that reduced face processing in social interactions in ASD is related to reduced sensitivity to the subtle expressive movements of a real face. These findings are consistent with face processing activity observed previously in right lateral prefrontal cortex using fMRI and TD participants [114]. Consistent with these prior findings, in the study reported here, the strongest negative correlation between ADOS-2 scores and neural activity (-0.77) was in this area and observed during eye contact (Figs 7 and 8). These findings suggest another target of further investigation in this dorsal neural pathway and its role in social performance.
Within this framework, our findings are consistent with the hypothesis that eye movements (increased positional variation) during live eye contact and social processing are altered in ASD and suggest that “bottom-up” factors may impact live eye-to-eye interactions. Altered incoming information due to visual sensing factors such as increased fixation variability could fail to capture higher-order motion cues associated with face and social interactions. Individuals with ASD show raised thresholds for the perception of coherent motion [5, 10, 115]. These findings also add further support for the “dorsal stream vulnerability” hypothesis in ASD suggesting that mechanisms supporting motion sensitivity such as live face interactions are compromised [46].
Limitations
The advantages of fNIRS are balanced by technical limitations relative to fMRI. The spatial resolution of fNIRS (approximately 3 cm) does not allow for discrimination of small anatomical differences in functional activity between gyri, and the origin of acquired signals does not extend below the superficial grey matter of the cortex of about 1.5–2.0 cm. Thus, findings of this, and other investigations based on fNIRS technology, are restricted to superficial cortical networks. Although the eye-to-eye contact occurs within a live context which is a novel advance, the gaze context in the study is relatively constrained which is necessary for experimental control and thus also sets constraints on the investigation of naturally occurring behaviors.
Conclusion
Despite recent advances in second-person neuroscience, the neurosocial mechanisms that underlie live face processing, a primary social communication system in humans, remains poorly understood. This knowledge gap is particularly profound in autism spectrum disorder where social difficulties are often disabling. Here we apply a two-person dynamic paradigm that incorporates oculomotor, neural, and social metrics as well as cross-brain coherence and individual differences to isolate neural biomarkers associated with reduced sensitivity to real-eye contact. Findings highlight the role of hypoactivity of right dorsal parietal and frontal systems in reduced sensitivity to live faces in ASD.
Supporting information
S1 Fig. Eye tracking report for participants with autism spectrum disorder (ASD), real eye condition.
Colors indicate the percentage of time eye gaze is within the eye region of the partner (dark blue = 0% and bright yellow = 100%) during each epoch of the time series (x-axis). The vertical axis includes all participants with ASD for whom eye tracking data were acquired.
https://doi.org/10.1371/journal.pone.0265798.s001
(TIF)
S2 Fig. Eye tracking report for typically-developed (TD) participants, real eye condition.
Colors indicate the percentage of time eye gaze is within the eye region of the partner (dark blue = 0% and bright yellow = 100%) during each epoch of the time series (x-axis). The vertical axis includes all TD participants for whom eye tracking data were acquired.
https://doi.org/10.1371/journal.pone.0265798.s002
(TIF)
S1 Table. Demographic information for participants with autism spectrum disorder (ASD).
Assessment measures include the Autism-Spectrum Quotient test (AQ, total scores); Broad Autism Phenotype Questionnaire (BAPQ, total scores); Social Responsiveness Scale, Second Edition (SRS-2, raw scores); Beck Anxiety Inventory (BAI, total scores); State-Trait Anxiety Inventory (STAI; total state anxiety scores); Liebowitz Social Anxiety Scale (LSAS, total scores); and the Autism Diagnostic Observation Schedule (ADOS-2, total scores). The Wechsler Abbreviated Scale of Intelligence, 2nd Edition (WASI-II) was administered to estimate full-scale intelligence quotient scores based on four subtests (FSIQ-4). *Indicates data were not acquired.
https://doi.org/10.1371/journal.pone.0265798.s003
(TIF)
S2 Table. Demographic information for typically-developed (TD) participants.
Assessment measures include the Autism-Spectrum Quotient test (AQ, total scores); Broad Autism Phenotype Questionnaire (BAPQ, total scores); Social Responsiveness Scale, Second Edition (SRS-2, raw scores); Beck Anxiety Inventory (BAI, total scores); State-Trait Anxiety Inventory (STAI; total state anxiety scores); and the Liebowitz Social Anxiety Scale (LSAS, total scores). The Wechsler Abbreviated Scale of Intelligence, 2nd Edition (WASI-II) was administered to estimate full-scale intelligence quotient scores based on four subtests (FSIQ-4). *Indicates data were not acquired.
https://doi.org/10.1371/journal.pone.0265798.s004
(TIF)
S3 Table. Comparison of participants with autism spectrum disorder (ASD) and typically-developed (TD) participant groups by gender, handedness, and age.
Groups were similar in terms of age and handedness; however, the ratio of male to female participants was higher in the ASD group than in the TD group. The gender composition of the ASD group is consistent with the estimated 4:1 male:female ratio of ASD diagnosis. This ratio increases to 6 males diagnosed with ASD for every 1 female in people whose cognitive functioning is within or above normal limits, such as those in our sample (Kirkovski, M., Enticott, P. G., & Fitzgerald, P. B. (2013). A review of the role of female gender in autism spectrum disorders. Journal of Autism and Developmental Disorders, 43(11), 2584–2603).
https://doi.org/10.1371/journal.pone.0265798.s005
(TIF)
S4 Table. Statistical comparisons (independent t-tests, two-tailed assuming unequal variances) of scores between typically-developed (TD) and autism spectrum disorder (ASD) groups are consistent with differences for the Autism-Spectrum Quotient test; Broad Autism phenotype questionnaire; social responsiveness scale, 2nd Edition; and the Beck Anxiety Inventory.
No evidence was found for differences between the groups for FSIQ-4 (estimated by the Wechsler Abbreviated Scale of Intelligence); State-Trait Anxiety Inventory (state anxiety items only); or the Liebowitz Social Anxiety Scale, and is taken as evidence in favor of matched groups with respect to these metrics.
https://doi.org/10.1371/journal.pone.0265798.s006
(TIF)
S5 Table.
Group averages and individual median percentages of eye-gaze time within the eye box of partners for typically developed (TD) participants (left column) and participants with autism spectrum disorder (ASD) (right column) during the Real Eye Condition. A t-test of these median percentages shows t(25) = 0.28 n.s. See S1 and S2 Figs for a graphical run-by-run representation of eye tracking performance.
https://doi.org/10.1371/journal.pone.0265798.s007
(TIF)
Acknowledgments
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The authors are grateful to the participants for their essential efforts to advance understanding of ASD; to our two lab partners, CD and IS, for consistent partnership with our participants and the investigators; and to Jen Cuzzocreo for database management and graphical representations of the data.
References
- 1.
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). Washington, D.C.: American Psychiatric Association; 2013.
- 2. Nation K, Penny S. Sensitivity to eye gaze in autism: Is it normal? Is it automatic? Is it social? Development and Psychopathology. 2008;20(1):79–97. pmid:18211729
- 3. Bookheimer SY, Wang AT, Scott A, Sigman M, Dapretto M. Frontal contributions to face processing differences in autism: Evidence from fMRI of inverted face processing. Journal of the International Neuropsychological Society. 2008;14(6):922–32. pmid:18954473
- 4. Constantino JN, Kennon-McGill S, Weichselbaum C, Marrus N, Haider A, Glowinski AL, et al. Infant viewing of social scenes is under genetic control and is atypical in autism. Nature. 2017;547(7663):340–4. pmid:28700580
- 5. Pelphrey KA, Morris JP, McCarthy G. Neural basis of eye gaze processing deficits in autism. Brain. 2005;128(5):1038–48. pmid:15758039
- 6. Schneier FR, Kent JM, Star A, Hirsch J. Neural circuitry of submissive behavior in social anxiety disorder: A preliminary study of response to direct eye gaze. Psychiatry Research: Neuroimaging. 2009;173(3):248–50. pmid:19628377
- 7. Senju A, Johnson MH. Atypical eye contact in autism: Models, mechanisms and development. Neuroscience & Biobehavioral Reviews. 2009;33(8):1204–14.
- 8. McPartland JC, Webb SJ, Keehn B, Dawson G. Patterns of visual attention to faces and objects in autism spectrum disorder. Journal of Autism and Developmental Disorders. 2011;41(2):148–57. pmid:20499148
- 9. Jones W, Klin A. Attention to eyes is present but in decline in 2-6-month-old infants later diagnosed with autism. Nature. 2013;504(7480):427. pmid:24196715
- 10. Pelphrey KA, Sasson NJ, Reznick JS, Paul G, Goldman BD, Piven J. Visual scanning of faces in autism. Journal of Autism and Developmental Disorders. 2002;32(4):249–61. pmid:12199131
- 11. Rahko JS, Paakki J-J, Starck TH, Nikkinen J, Pauls DL, Kätsyri JV, et al. Valence scaling of dynamic facial expressions is altered in high-functioning subjects with autism spectrum disorders: an fMRI study. Journal of autism and developmental disorders. 2012;42(6):1011–24. pmid:21822763
- 12. Chawarska K, Klin A, Volkmar F. Automatic attention cueing through eye movement in 2-year-old children with autism. Child Development. 2003;74(4):1108–22. pmid:12938707
- 13. Uljarevic M, Hamilton A. Recognition of emotions in autism: a formal meta-analysis. Journal of autism and developmental disorders. 2013;43(7):1517–26. pmid:23114566
- 14. Campbell R, Lawrence K, Mandy W, Mitra C, Jeyakuma L, Skuse D. Meanings in motion and faces: Developmental associations between the processing of intention from geometrical animations and gaze detection accuracy. Developmental Psychopathology. 2006;18(1):99–118. pmid:16478554
- 15. Dawson G, Webb SJ, McPartland J. Understanding the nature of face processing impairment in autism: Insights from behavioral and electrophysiological studies. Developmental Neuropsychology. 2005;27(3):403–24. pmid:15843104
- 16. Tanabe HC, Kosaka H, Saito DN, Koike T, Hayashi MJ, Izuma K, et al. Hard to “tune in”: Neural mechanisms of live face-to-face interaction with high-functioning autistic spectrum disorder. Frontiers in Human Neuroscience. 2012;6:268. pmid:23060772
- 17. Tottenham N, Hertzig ME, Gillespie-Lynch K, Gilhooly T, Millner AJ, Casey B. Elevated amygdala response to faces and gaze aversion in autism spectrum disorder. Social Cognitive and Affective Neuroscience. 2014;9(1):106–17. pmid:23596190
- 18. McPartland J, Cheung CH, Perszyk D, Mayes LC. Face-related ERPs are modulated by point of gaze. Neuropsychologia. 2010;48(12):3657–60. pmid:20654637
- 19. McPartland J, Dawson G, Webb SJ, Panagiotides H, Carver LJ. Event-related brain potentials reveal anomalies in temporal processing of faces in autism spectrum disorder. Journal of Child Psychology and Psychiatry. 2004;45(7):1235–45. pmid:15335344
- 20. Parker TC, Crowley MJ, Naples AJ, Rolison MJ, Wu J, Trapani JA, et al. The N170 event-related potential reflects delayed neural response to faces when visual attention is directed to the eyes in youths with ASD. Autism Research. 2021;14:1–10.
- 21. Jung CE, Strother L, Feil-Seifer DJ, Hutsler JJ. Atypical asymmetry for processing human and robot faces in autism revealed by fNIRS. PLoS One. 2016;11(7):e0158804. pmid:27389017
- 22. Puce A, Allison T, Bentin S, Gore JC, McCarthy G. Temporal cortex activation in humans viewing eye and mouth movements. Journal of neuroscience. 1998;18(6):2188–99. pmid:9482803
- 23. Puce A, Allison T, McCarthy G. Electrophysiological studies of human face perception. III: Effects of top-down processing on face-specific potentials. Cerebral cortex. 1999;9(5):445–58. pmid:10450890
- 24. Allison T, Puce A, McCarthy G. Social perception from visual cues: role of the STS region. Trends in cognitive sciences. 2000;4(7):267–78. pmid:10859571
- 25. Hadjikhani N, Joseph RM, Snyder J, Tager-Flusberg H. Abnormal activation of the social brain during face perception in autism. Human Brain Mapping. 2007;28(5):441–9. pmid:17133386
- 26. Redcay E, Saxe R. Do you see what I see? The neural bases of joint attention. Agency and Joint Attention. 2013:216.
- 27. Harris GJ, Chabris CF, Clark J, Urban T, Aharon I, Steele S, et al. Brain activation during semantic processing in autism spectrum disorders via functional magnetic resonance imaging. Brain and cognition. 2006;61(1):54–68. pmid:16473449
- 28. Lai G, Pantazatos SP, Schneider H, Hirsch J. Neural systems for speech and song in autism. Brain. 2012;135(3):961–75.
- 29. Golarai G, Grill-Spector K, Reiss AL. Autism and the development of face processing. Clinical Neuroscience Research. 2006;6(3–4):145–60. pmid:18176635
- 30. Etkin A, Klemenhagen KC, Dudman JT, Rogan MT, Hen R, Kandel ER, et al. Individual differences in trait anxiety predict the response of the basolateral amygdala to unconsciously processed fearful faces. Neuron. 2004;44(6):1043–55. pmid:15603746
- 31. Trevisan DA, Roberts N, Lin C, Birmingham E. How do adults and teens with self-declared Autism Spectrum Disorder experience eye contact? A qualitative analysis of first-hand accounts. PLoS ONE. 2017;12(11):e0188446. pmid:29182643
- 32. Hasson U, Frith CD. Mirroring and beyond: Coupled dynamics as a generalized framework for modelling social interactions. Philosophical Transactions of the Royal Society B: Biological Sciences. 2016;371(1693):20150366. pmid:27069044
- 33. Redcay E, Schilbach L. Using second-person neuroscience to elucidate the mechanisms of social interaction. Nature Reviews Neuroscience. 2019;20(8):495–505. pmid:31138910
- 34. Schilbach L, Timmermans B, Reddy V, Costall A, Bente G, Schlicht T, et al. Toward a second-person neuroscience. Behavioral and Brain Sciences. 2013;36(04):393–414.
- 35. Schilbach L. Towards a second-person neuropsychiatry. Philosophical Transactions of the Royal Society B: Biological Sciences. 2016;371(1686):20150081. pmid:26644599
- 36.
Schilbach L. Using interaction-based phenotyping to assess the behavioral and neural mechanisms of transdiagnostic social impairments in psychiatry. Springer; 2019. p. 273–4.
- 37. Bolis D, Balsters J, Wenderoth N, Becchio C, Schilbach L. Beyond Autism: Introducing the Dialectical Misattunement Hypothesis and a Bayesian Account of Intersubjectivity. Psychopathology. 2017;50(6):355–72. pmid:29232684
- 38. Bolis D, Schilbach L. Observing and participating in social interactions: Action perception and action control across the autistic spectrum. Dev Cogn Neuros-Neth. 2018;29:168–75. pmid:28188104
- 39. Rolison MJ, Naples AJ, McPartland JC. Interactive social neuroscience to study autism spectrum disorder. The Yale Journal of Biology and Medicine. 2015;88(1):17–24. pmid:25745371
- 40.
Boas DA, Elwell CE, Ferrari M, Taga G. Twenty years of functional near-infrared spectroscopy: introduction for the special issue. Elsevier; 2014.
- 41. Ferrari M, Quaresima V. A brief review on the history of human functional near-infrared spectroscopy (fNIRS) development and fields of application. Neuroimage. 2012;63(2):921–35. pmid:22510258
- 42. Carter RM, Huettel SA. A nexus model of the temporal-parietal junction. Trends in Cognitive Sciences. 2013;17(7):328–36. pmid:23790322
- 43. Noah JA, Dravida S, Zhang X, Yahil S, Hirsch J. Neural correlates of conflict between gestures and words: A domain-specific role for a temporal-parietal complex. PLoS ONE. 2017;12(3):e0173525. pmid:28278240
- 44. Kelley M, Noah JA, Zhang X, Scassellati B, Hirsch J. Comparison of human social brain activity during eye-contact with another human and a humanoid robot. Frontiers in Robotics and AI. 2021;7:599581. pmid:33585574
- 45. Noah JA, Zhang X, Dravida S, Ono Y, Naples A, McPartland JC, et al. Real-time eye-to-eye contact is associated with cross-brain neural coupling in angular gyrus. Frontiers in Human Neuroscience. 2020;14:19. pmid:32116606
- 46. Braddick O, Atkinson J, Wattam-Bell J. Normal and anomalous development of visual motion processing: motion coherence and ‘dorsal-stream vulnerability’. Neuropsychologia. 2003;41(13):1769–84.
- 47.
Lord C, Rutter M, DiLavore P, Risi S, Gotham K, Bishop S. Autism Diagnostic Observation Schedule. Western Psychological Services. Los Angeles, CA. 2012.
- 48.
Constantino JN, Gruber CP. Social responsiveness scale, second edition (SRS-2): Manual. Los Angeles: CA: Western Psychological Services 2012.
- 49. Oldfield RC. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia. 1971;9(1):97–113. pmid:5146491
- 50. Champely S, Ekstrom C, Dalgaard P, Gill J, Weibelzahl S, Anandkumar A, et al. pwr: Basic functions for power analysis. 2017.
- 51.
Wechsler D. WASI-II: Wechsler Abbreviated Scale of Intelligence®, Second Edition. San Antonio, TX: NCS Pearson; 2011.
- 52. Baron-Cohen S, Wheelwright S, Skinner R, Martin J, Clubley E. The autism-spectrum quotient (AQ): Evidence from asperger syndrome/high-functioning autism, males and females, scientists, and mathematicians. Journal of Autism and Developmental Disorders. 2001;31(1):5–17. pmid:11439754
- 53. Hurley RS, Losh M, Parlier M, Reznick JS, Piven J. The broad autism phenotype questionnaire. Journal of Autism and Developmental Disorders. 2007;37(9):1679–90. pmid:17146701
- 54. Beck AT, Steer RA. Relationship between the Beck anxiety inventory and the Hamilton anxiety rating scale with anxious outpatients. Journal of Anxiety Disorders. 1991;5(3):213–23.
- 55.
Spielberger C, Gorsuch R, Lushene R. State-trait anxiety inventory (STAI). Palo Alto, CA: Mind Garden; 1983.
- 56. Fresco D, Coles M, Heimberg RG, Liebowitz M, Hami S, Stein MB, et al. The Liebowitz Social Anxiety Scale: A comparison of the psychometric properties of self-report and clinician-administered formats. Psychological Medicine. 2001;31(6):1025–35. pmid:11513370
- 57. Falck-Ytter T. Gaze performance during face-to-face communication: A live eye tracking study of typical children and children with autism. Research in Autism Spectrum Disorders. 2015;17:78–85.
- 58. Thorup E, Nyström P, Gredebäck G, Bölte S, Falck-Ytter T. Altered gaze following during live interaction in infants at risk for autism: an eye tracking study. Molecular autism. 2016;7(1):1–10.
- 59. Dravida S, Noah JA, Zhang X, Hirsch J. Joint attention during live person-to-person contact activates rTPJ, including a sub-component associated with spontaneous eye-to-eye contact. Frontiers in Human Neuroscience. 2020;14.
- 60. Zhang X, Noah JA, Hirsch J. Separation of the global and local components in functional near-infrared spectroscopy signals using principal component spatial filtering. Neurophotonics. 2016;3(1):015004. pmid:26866047
- 61. Noah JA, Ono Y, Nomoto Y, Shimada S, Tachibana A, Zhang X, et al. fMRI validation of fNIRS measurements during a naturalistic task. Journal of Visualized Experiments: JoVE. 2015(100). pmid:26132365
- 62. Zhang X, Noah JA, Dravida S, Hirsch J. Signal processing of functional NIRS data acquired during overt speaking. Neurophotonics. 2017;4(4):041409. pmid:28924564
- 63. Dravida S, Noah JA, Zhang X, Hirsch J. Comparison of oxyhemoglobin and deoxyhemoglobin signal reliability with and without global mean removal for digit manipulation motor tasks. Neurophotonics. 2018;5(1):011006. pmid:28924566
- 64. Hirsch J, Noah JA, Zhang X, Dravida S, Ono Y. A cross-brain neural mechanism for human-to-human verbal communication. Social Cognitive and Affective Neuroscience. 2018;13(9):907–20. pmid:30137601
- 65. Piva M, Zhang X, Noah JA, Chang SW, Hirsch J. Distributed neural activity patterns during human-to-human competition. Frontiers in Human Neuroscience. 2017;11:571. pmid:29218005
- 66. Ono Y, Nomoto Y, Tanaka S, Sato K, Shimada S, Tachibana A, et al. Frontotemporal oxyhemoglobin dynamics predict performance accuracy of dance simulation gameplay: Temporal characteristics of top-down and bottom-up cortical activities. NeuroImage. 2014;85:461–70. pmid:23707582
- 67. Tachibana A, Noah JA, Bronner S, Ono Y, Onozuka M. Parietal and temporal activity during a multimodal dance video game: An fNIRS study. Neuroscience Letters. 2011;503(2):125–30. pmid:21875646
- 68. Okamoto M, Dan I. Automated cortical projection of head-surface locations for transcranial functional brain mapping. NeuroImage. 2005;26(1):18–28. pmid:15862201
- 69. Singh AK, Okamoto M, Dan H, Jurcak V, Dan I. Spatial registration of multichannel multi-subject fNIRS data to MNI space without MRI. NeuroImage. 2005;27(4):842–51. pmid:15979346
- 70. Eggebrecht AT, Ferradal SL, Robichaux-Viehoever A, Hassanpour MS, Dehghani H, Snyder AZ, et al. Mapping distributed brain function and networks with diffuse optical tomography. Nature Photonics. 2014;8(6):448. pmid:25083161
- 71. Ferradal SL, Eggebrecht AT, Hassanpour M, Snyder AZ, Culver JP. Atlas-based head modeling and spatial normalization for high-density diffuse optical tomography: In vivo validation against fMRI. NeuroImage. 2014;85:117–26. pmid:23578579
- 72. Eggebrecht AT, White BR, Ferradal SL, Chen C, Zhan Y, Snyder AZ, et al. A quantitative spatial comparison of high-density diffuse optical tomography and fMRI cortical mapping. NeuroImage. 2012;61(4):1120–8. pmid:22330315
- 73. Mazziotta J, Toga A, Evans A, Fox P, Lancaster J, Zilles K, et al. A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). Philosophical Transactions of the Royal Society B: Biological Sciences. 2001;356(1412):1293–322.
- 74. Ye JC, Tak S, Jang KE, Jung J, Jang J. NIRS-SPM: statistical parametric mapping for near-infrared spectroscopy. NeuroImage. 2009;44(2):428–47. pmid:18848897
- 75. Maldjian JA, Laurienti PJ, Kraft RA, Burdette JH. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. Neuroimage. 2003;19(3):1233–9. pmid:12880848
- 76. Maldjian JA, Laurienti PJ, Burdette JH. Precentral gyrus discrepancy in electronic versions of the Talairach atlas. Neuroimage. 2004;21(1):450–5. pmid:14741682
- 77. Hazeki O, Tamura M. Quantitative analysis of hemoglobin oxygenation state of rat brain in situ by near-infrared spectrophotometry. Journal of Applied Physiology. 1988;64(2):796–802. pmid:3372437
- 78. Hoshi Y. Functional near-infrared optical imaging: Utility and limitations in human brain mapping. Psychophysiology. 2003;40(4):511–20. pmid:14570159
- 79. Matcher SJ, Elwell CE, Cooper CE, Cope M, Delpy DT. Performance comparison of several published tissue near-infrared spectroscopy algorithms. Anal Biochem. 1995;227(1):54–68. pmid:7668392
- 80. Yücel M, Lühmann A, Scholkmann F, Gervain J, Dan I, Ayaz H, et al. Best practices for fNIRS publications. Neurophotonics. 2021;8(1):012101. pmid:33442557
- 81. Tachtsidis I, Scholkmann F. False positives and false negatives in functional near-infrared spectroscopy: Issues, challenges, and the way forward. Neurophotonics. 2016;3(3):031405. pmid:27054143
- 82. Noah JA, Zhang X, Dravida S, DiCocco C, Suzuki T, Aslin RN, et al. Comparison of short-channel separation and spatial domain filtering for removal of non-neural components in functional near-infrared spectroscopy signals. Neurophotonics. 2021;8(1):015004. pmid:33598505
- 83.
Tachtsidis I, Tisdall MM, Leung TS, Pritchard C, Cooper CE, Smith M, et al. Relationship between brain tissue haemodynamics, oxygenation and metabolism in the healthy human adult brain during hyperoxia and hypercapnea. Oxygen Transport to Tissue XXX: Springer; 2009. p. 315–20.
- 84.
Penny WD, Friston KJ, Ashburner JT, Kiebel SJ, Nichols TE. Statistical parametric mapping: The analysis of functional brain images: Elsevier; 2011.
- 85. Hirsch J, Zhang X, Noah JA, Ono Y. Frontal temporal and parietal systems synchronize within and across brains during live eye-to-eye contact. NeuroImage. 2017;157:314–30. pmid:28619652
- 86. Cui X, Bryant DM, Reiss AL. NIRS-based hyperscanning reveals increased interpersonal coherence in superior frontal cortex during cooperation. Neuroimage. 2012;59(3):2430–7. pmid:21933717
- 87. Torrence C, Compo GP. A practical guide to wavelet analysis. Bulletin of the American Meteorological society. 1998;79(1):61–78.
- 88. Friston K, Buechel C, Fink G, Morris J, Rolls E, Dolan RJ. Psychophysiological and modulatory interactions in neuroimaging. Neuroimage. 1997;6(3):218–29. pmid:9344826
- 89. de Vries L, Fouquaet I, Boets B, Naulaers G, Steyaert J. Autism spectrum disorder and pupillometry: A systematic review and meta-analysis. Neuroscience & Biobehavioral Reviews. 2021;120:479–508. pmid:33172600
- 90. Beatty J. Task-Evoked Pupillary Responses, Processing Load, and the Structure of Processing Resources. Psychol Bull. 1982;91(2):276–92. pmid:7071262
- 91. Schroeder CE, Wilson DA, Radman T, Scharfman H, Lakatos P. Dynamics of active sensing and perceptual selection. Current Opinion in Neurobiology. 2010;20(2):172–6. pmid:20307966
- 92. Zweifel NO, Hartmann MJ. Defining “active sensing” through an analysis of sensing energetics: Homeoactive and alloactive sensing. Journal of Neurophysiology. 2020;124(1):40–8. pmid:32432502
- 93. Simmons DR, Robertson AE, McKay LS, Toal E, McAleer P, Pollick FE. Vision in autism spectrum disorders. Vision Research. 2009;49(22):2705–39. pmid:19682485
- 94. Goldberg M, Lasker A, Zee D, Garth E, Tien A, Landa R. Deficits in the initiation of eye movements in the absence of a visual target in adolescents with high functioning autism. Neuropsychologia. 2002;40(12):2039–49. pmid:12208001
- 95. Haxby JV, Hoffman EA, Gobbini MI. The distributed human neural system for face perception. Trends in Cognitive Sciences. 2000;4(6):223–33. pmid:10827445
- 96. Kanwisher N, McDermott J, Chun MM. The fusiform face area: A module in human extrastriate cortex specialized for face perception. Journal of Neuroscience. 1997;17(11):4302–11. pmid:9151747
- 97. Chang L, Tsao DY. The code for facial identity in the primate brain. Cell. 2017;169(6):1013–28. e14. pmid:28575666
- 98. Johnson MH, Griffin R, Csibra G, Halit H, Farroni T, de Haan M, et al. The emergence of the social brain network: Evidence from typical and atypical development. Development and Psychopathology. 2005;17(03):599–619. pmid:16262984
- 99. Scholkmann F, Holper L, Wolf U, Wolf M. A new methodical approach in neuroscience: Assessing inter-personal brain coupling using functional near-infrared imaging (fNIRI) hyperscanning. Frontiers in Human Neuroscience. 2013;7:813. pmid:24348362
- 100. Babiloni F, Astolfi L. Social neuroscience and hyperscanning techniques: Past, present and future. Neuroscience & Biobehavioral Reviews. 2014;44:76–93. pmid:22917915
- 101. Bilek E, Ruf M, Schäfer A, Akdeniz C, Calhoun VD, Schmahl C, et al. Information flow between interacting human brains: Identification, validation, and relationship to social expertise. Proceedings of the National Academy of Sciences. 2015;112(16):5207–12.
- 102. Wheatley T, Boncz A, Toni I, Stolk A. Beyond the isolated brain: the promise and challenge of interacting minds. Neuron. 2019;103(2):186–8. pmid:31319048
- 103. Kingsbury L, Hong W. A multi-brain framework for social interaction. Trends in Neurosciences. 2020;43(9):651–66. pmid:32709376
- 104. Hasson U, Ghazanfar AA, Galantucci B, Garrod S, Keysers C. Brain-to-brain coupling: A mechanism for creating and sharing a social world. Trends in Cognitive Sciences. 2012;16(2):114–21. pmid:22221820
- 105. De Jaegher H, Di Paolo E, Adolphs R. What does the interactive brain hypothesis mean for social neuroscience? A dialogue. Philosophical Transactions of the Royal Society B: Biological Sciences. 2016;371(1693):20150379. pmid:27069056
- 106. Di Paolo EA, De Jaegher H. The interactive brain hypothesis. Frontiers in Human Neuroscience. 2012;6.
- 107. Redcay E, Kleiner M, Saxe R. Look at this: The neural correlates of initiating and responding to bids for joint attention. Frontiers in Human Neuroscience. 2012;6:169. pmid:22737112
- 108. Huppert TJ, Hoge RD, Diamond SG, Franceschini MA, Boas DA. A temporal comparison of BOLD, ASL, and NIRS hemodynamic responses to motor stimuli in adult humans. Neuroimage. 2006;29(2):368–82. pmid:16303317
- 109. Jöbsis FF. Noninvasive, infrared monitoring of cerebral and myocardial oxygen sufficiency and circulatory parameters. Science. 1977;198(4323):1264–7. pmid:929199
- 110. Strangman G, Culver JP, Thompson JH, Boas DA. A quantitative comparison of simultaneous BOLD fMRI and NIRS recordings during functional brain activation. Neuroimage. 2002;17(2):719–31. pmid:12377147
- 111. Villringer A, Chance B. Non-invasive optical spectroscopy and imaging of human brain function. Trends in Neurosciences. 1997;20(10):435–42. pmid:9347608
- 112. Scherf KS, Elbich D, Minshew N, Behrmann M. Individual differences in symptom severity and behavior predict neural activation during face processing in adolescents with autism. NeuroImage: Clinical. 2015;7:53–67. pmid:25610767
- 113. McPartland JC. Developing clinically practicable biomarkers for autism spectrum disorder. Journal of autism and developmental disorders. 2017;47(9):2935–7. pmid:28695438
- 114. Chan AW, Downing PE. Faces and eyes in human lateral prefrontal cortex. Frontiers in human neuroscience. 2011;5:51. pmid:21687796
- 115. Happé F, Frith U. The weak coherence account: detail-focused cognitive style in autism spectrum disorders. Journal of autism and developmental disorders. 2006;36(1):5–25. pmid:16450045