Chapter 1: Introduction
Our visual experience generally takes spatial contexts into account. This process of contextual modulation not only enables the function of object segregation but also alters the phenomenological appearance of objects. For instance, perceptual constancy and some types of visual illusions are biologically established as a result of contextual modulation. Orientation repulsion is such an example of phenomenological change induced by a spatial context: the orientation of a target is apparently biased away from that of a surrounding inducer (e.g., Clifford, 2014). Although previous studies have extensively investigated the static properties of this phenomenon, little is known about its dynamic formation process.
The suggestion that contextual modulation for orientation repulsion can be a sluggish process stems from its involvement of multiple stages in the visual processing hierarchy, its activation of various neuronal circuits (e.g., horizontal propagation within the primary visual cortex, V1, and feedback pathways from extrastriate cortices), and its requirement of complex computations (divisive normalization; see e.g., Goddard et al., 2008). While the dynamic formation process has been predominantly inferred using neurophysiological methods, their lack of referencing to conscious reports limits the researchers’ ability to probe the preconscious content, which has the potential to arise in conscious awareness.
Therefore, the overall aim of my dissertation is to uncover the dynamic formation process preceding the awareness of a target in orientation repulsion, utilizing various psychophysical techniques that manipulate the temporal processing of vision. In each chapter, I examined how the size of repulsion was altered by backward masking (Chapter 2), facilitation by a preceding flanker (Chapter 3), temporal expectation (Chapter 4), inducer duration (Chapter 5), and spatiotemporal distance from the inducer (Chapter 6).
Chapter 2: Study 1―Backward masking
In Study 1, I aimed to examine how the size of orientation repulsion was altered when backward masking terminated the processing for the target in repulsion. Backward masking occurs when a briefly flashed visual stimulus (called a target) is rendered less visible due to another stimulus (called a mask) delivered right after the target (e.g., Breitmeyer & Öğmen, 2006; Di Lollo et al., 2000). I compared the size of repulsion between the two conditions explained below. In the simultaneous-offset condition, a target Gabor patch having a vertical orientation and a surrounding four-dot mask were briefly flashed and disappeared together; thus, backward masking was not expected to occur. In the delayed-offset condition, the mask appeared together with the target, but remained there for 300 milliseconds (ms) after the target disappeared; thus, backward masking was expected to occur. In both conditions, eight tilted Gabor patches were arranged circularly around the target, serving as inducers for the orientation repulsion. Observers were asked to indicate whether the target appeared tilted clockwise or counter-clockwise from the vertical.
I found that backward masking reduced orientation repulsion while orientation discriminability was left intact (Experiment 1). Furthermore, the reduction in repulsion was caused only by inducers presented simultaneously with or after the target but not by preceding inducers (Experiment 2). These findings suggest the dynamic formation process of orientation repulsion: a vertical target is initially represented as vertical, and the internal representation temporally evolves to be tilted against the orientation of inducers prior to conscious awareness. In case common-onset masking terminates the temporal evolution, observers are forced to be aware of a premature representation corresponding to weaker repulsion.
Chapter 3: Study 2―Facilitation by preceding stimuli
In Study 2, I aimed to examine how the size of repulsion is altered by quickening the moment of awareness. Capitalizing on the phenomena of attentional prior entry (e.g., Shore et al., 2001) and perceptual latency priming (e.g., Scharlau, 2007), the awareness of a target in repulsion was artificially quickened by a flanker that preceded the target. Using the same spatial configuration as Study 1, I compared the size of repulsion between when the flanker preceded the target by 100 ms and when they appeared simultaneously.
I found that the preceding flanker reduced repulsion whether or not it was informative about the location of the impending target (Experiment 3). In contrast to the effect of the flanker timing, endogenous orienting of spatial attention by a central cue did not alter the size of repulsion whereas it improved the speeded performance of an orientation discrimination task (Experiment 4). In the stimulus configuration I used, I confirmed that the flanker indeed quickened the awareness of a locally succeeding target by approximately 40 ms (Experiment 5). Furthermore, I found that the flanker that was greater than 7 degrees away from the target also reduced repulsion when it preceded the target by 100 ms (Experiment 6). These findings suggest that the process controlling the moment of awareness operates independently of the process forming the perceptual content and that the former can finalize the appearance of repulsion even when contextual modulation has not yet been completed. In contrast, the findings are inconsistent with the view that the content of awareness is finalized when the content formation process reaches a certain steady state (such as when prediction error is minimized; see e.g., Di Lollo et al., 2000).
Chapter 4: Study 3―Develpiong temporal expectation
In Study 3, I aimed to examine whether the temporal information carried by preceding stimuli is crucial for altering the repulsion. I manipulated the predictability of the target onset and quantified its effect on repulsion. Prior to the target, an audiovisual cue uninformative of the target orientation and location was presented. There were three levels of foreperiods between the cue and target but with the highest probability of the middle foreperiod being chosen. The spatial configuration was the same as in Study 1.
I found that repulsion was reduced when the target appeared at or later than the moment predicted by a temporal cue (Experiment 7). Nonetheless, the rhythmic structure of cues alone failed to modify repulsion, at least under the condition that observers were unaware of the temporal contingency between the cue and target (Experiment 8). Furthermore, when the foreperiod between a single temporal cue and the target was manipulated within an experimental session, repulsion decreased as the foreperiod became longer (Experiment 9). Based on the evidence in previous studies showing that temporal expectation hastens the decision onset (e.g., Devine et al, 2019), the findings suggest that developing temporal expectation enables premature accumulation of an evolving perceptual content, leading to a weaker repulsion.
Chapter 5: Study 4―Rapid stimulus alternation
In Study 4, I aimed to quantify the temporal resolution and temporal extent of orientation repulsion in the same paradigm. Temporal resolution and temporal extent are operationally defined as the minimal time to induce repulsion and the maximal time to increase repulsion, respectively. I used an alternating pair of inducer stimuli: one tilted 15° (optimized to induce the maximal repulsion) and the other had a complementary orientation distribution, effectively fused into a concentric pattern with rapid alternation. I quantified repulsion at various alternation frequencies. The constraint that a vertical target was briefly flashed at the midpoint of the duration of the 15° inducer allowed the time window for potential contextual modulation to widen systematically with decreasing frequency.
When an orthogonal pair of D2 patterns, a type of grating whose luminance modulation in a particular orientation was the second-order partial derivative of an isotropic 2D-Gaussian (introduced by Motoyoshi & Nishida, 2001), was used as the inducer, repulsion increased as inducer duration increased from 20 to 30 ms and then leveled off (Experiment 10). When a complementary pair of custom-made textures was used as the inducer, repulsion continued to increase until the duration reached 200 ms (Experiment 11). This gradual increase of repulsion was observed regardless of conscious accessibility to the orientation of the inducer accompanying the target (Experiment 12). I also confirmed that, to quantify the temporal resolution of repulsion, an alternating pair of inducer stimuli must either cause opposing illusions or fuse into a pattern having no orientation bias because repulsion always occurred regardless of inducer orientation if neither condition was met (Experiment 13). These findings reveal that contextual modulation in orientation occurs at a high temporal resolution and continues to a long temporal extent, corroborating my view that perceptual content evolves prior to the awareness of a target in orientation repulsion.
Chapter 6: Study 5―Reverse correlation
In Study 5, I aimed to examine how the dynamic formation process of orientation repulsion depends on spatial distance. Using a psychophysical reverse correlation method, a previous study estimated the temporal kernel of inducing stimuli contributing to repulsion as within 100 ms around the target (Mareschal & Clifford, 2012). Extending this paradigm, I estimated the peak time of the temporal kernel at three different spatial distances between the target and inducing stimuli. Given that modulatory signals from a more distant surround should take more time to be transmitted, the peak time was predicted to shift earlier into the past as the inducer became farther from the target.
However, the result was the opposite. I found that as the spatial distance became longer, the peak time of the temporal kernel shifted into the future relative to the target (Experiment 14). To account for this counterintuitive finding, I suggest that contextual modulation is an active process: it may start with query-like signals sent to the surround after detecting a behaviorally relevant target rather than the surround automatically sending modulatory signals regardless of what stands as the target. This “querying” process inevitably lags behind the direct input, and the difference in the lag at different spatial distances may underlie the temporal evolution of orientation repulsion.
Chapter 7: General discussion
Overall, my dissertation provides novel insights into the dynamic formation process of orientation repulsion. I have demonstrated that repulsion builds up with extending inducer duration (Chapter 5) while querying the broad spatial extent of surrounds about the contextual information (Chapter 6). When this dynamic formation process was prematurely terminated (Chapter 2), facilitated in a bottom-up (Chapter 3) or top-down (Chapter 4) manner, the perceptual content incompletely modulated by spatial contexts arises in the awareness, giving a weaker repulsion.
All the findings consistently suggest that content of orientation repulsion undergoes a gradual temporal evolution, yet observers can consciously access the final output of this process. I consider that the dynamics of contextual modulation for repulsion may be realized by the iso-orientation surround suppression, especially operated in V1 (e.g., DeAngelis et al., 1994). Assuming that the orientation tuning for suppression signals sharpens over time (see Ringach et al., 1997), the tuning for the target will gradually shift against the orientation of inducers. Furthermore, the findings about temporal facilitation effects on repulsion (Chapters 3 and 4) support the view that the content of awareness is constructed as a whole-brain process beyond the visual cortex. I speculate that the premature content may have functional significance in conditions where a speeded reaction is more critical than creating an accurate representation, such as in the conditions of object localization and saliency computation.
The size of repulsion is advantageous among various measures of consciousness because it focuses on definitely suprathreshold content and is immune to response bias. Such a robust measure of contextual phenomena may also contribute to a deeper understanding of symptoms in clinical disorders.
To conclude, contextual modulation underlying the awareness of a target in orientation repulsion is a temporally unfolding process that involves multiple mechanisms with a range of spatiotemporal properties and can be probed with various psychophysical methodologies I introduced in this dissertation.
Our visual experience generally takes spatial contexts into account. This process of contextual modulation not only enables the function of object segregation but also alters the phenomenological appearance of objects. For instance, perceptual constancy and some types of visual illusions are biologically established as a result of contextual modulation. Orientation repulsion is such an example of phenomenological change induced by a spatial context: the orientation of a target is apparently biased away from that of a surrounding inducer (e.g., Clifford, 2014). Although previous studies have extensively investigated the static properties of this phenomenon, little is known about its dynamic formation process.
The suggestion that contextual modulation for orientation repulsion can be a sluggish process stems from its involvement of multiple stages in the visual processing hierarchy, its activation of various neuronal circuits (e.g., horizontal propagation within the primary visual cortex, V1, and feedback pathways from extrastriate cortices), and its requirement of complex computations (divisive normalization; see e.g., Goddard et al., 2008). While the dynamic formation process has been predominantly inferred using neurophysiological methods, their lack of referencing to conscious reports limits the researchers’ ability to probe the preconscious content, which has the potential to arise in conscious awareness.
Therefore, the overall aim of my dissertation is to uncover the dynamic formation process preceding the awareness of a target in orientation repulsion, utilizing various psychophysical techniques that manipulate the temporal processing of vision. In each chapter, I examined how the size of repulsion was altered by backward masking (Chapter 2), facilitation by a preceding flanker (Chapter 3), temporal expectation (Chapter 4), inducer duration (Chapter 5), and spatiotemporal distance from the inducer (Chapter 6).
Chapter 2: Study 1―Backward masking
In Study 1, I aimed to examine how the size of orientation repulsion was altered when backward masking terminated the processing for the target in repulsion. Backward masking occurs when a briefly flashed visual stimulus (called a target) is rendered less visible due to another stimulus (called a mask) delivered right after the target (e.g., Breitmeyer & Öğmen, 2006; Di Lollo et al., 2000). I compared the size of repulsion between the two conditions explained below. In the simultaneous-offset condition, a target Gabor patch having a vertical orientation and a surrounding four-dot mask were briefly flashed and disappeared together; thus, backward masking was not expected to occur. In the delayed-offset condition, the mask appeared together with the target, but remained there for 300 milliseconds (ms) after the target disappeared; thus, backward masking was expected to occur. In both conditions, eight tilted Gabor patches were arranged circularly around the target, serving as inducers for the orientation repulsion. Observers were asked to indicate whether the target appeared tilted clockwise or counter-clockwise from the vertical.
I found that backward masking reduced orientation repulsion while orientation discriminability was left intact (Experiment 1). Furthermore, the reduction in repulsion was caused only by inducers presented simultaneously with or after the target but not by preceding inducers (Experiment 2). These findings suggest the dynamic formation process of orientation repulsion: a vertical target is initially represented as vertical, and the internal representation temporally evolves to be tilted against the orientation of inducers prior to conscious awareness. In case common-onset masking terminates the temporal evolution, observers are forced to be aware of a premature representation corresponding to weaker repulsion.
Chapter 3: Study 2―Facilitation by preceding stimuli
In Study 2, I aimed to examine how the size of repulsion is altered by quickening the moment of awareness. Capitalizing on the phenomena of attentional prior entry (e.g., Shore et al., 2001) and perceptual latency priming (e.g., Scharlau, 2007), the awareness of a target in repulsion was artificially quickened by a flanker that preceded the target. Using the same spatial configuration as Study 1, I compared the size of repulsion between when the flanker preceded the target by 100 ms and when they appeared simultaneously.
I found that the preceding flanker reduced repulsion whether or not it was informative about the location of the impending target (Experiment 3). In contrast to the effect of the flanker timing, endogenous orienting of spatial attention by a central cue did not alter the size of repulsion whereas it improved the speeded performance of an orientation discrimination task (Experiment 4). In the stimulus configuration I used, I confirmed that the flanker indeed quickened the awareness of a locally succeeding target by approximately 40 ms (Experiment 5). Furthermore, I found that the flanker that was greater than 7 degrees away from the target also reduced repulsion when it preceded the target by 100 ms (Experiment 6). These findings suggest that the process controlling the moment of awareness operates independently of the process forming the perceptual content and that the former can finalize the appearance of repulsion even when contextual modulation has not yet been completed. In contrast, the findings are inconsistent with the view that the content of awareness is finalized when the content formation process reaches a certain steady state (such as when prediction error is minimized; see e.g., Di Lollo et al., 2000).
Chapter 4: Study 3―Develpiong temporal expectation
In Study 3, I aimed to examine whether the temporal information carried by preceding stimuli is crucial for altering the repulsion. I manipulated the predictability of the target onset and quantified its effect on repulsion. Prior to the target, an audiovisual cue uninformative of the target orientation and location was presented. There were three levels of foreperiods between the cue and target but with the highest probability of the middle foreperiod being chosen. The spatial configuration was the same as in Study 1.
I found that repulsion was reduced when the target appeared at or later than the moment predicted by a temporal cue (Experiment 7). Nonetheless, the rhythmic structure of cues alone failed to modify repulsion, at least under the condition that observers were unaware of the temporal contingency between the cue and target (Experiment 8). Furthermore, when the foreperiod between a single temporal cue and the target was manipulated within an experimental session, repulsion decreased as the foreperiod became longer (Experiment 9). Based on the evidence in previous studies showing that temporal expectation hastens the decision onset (e.g., Devine et al, 2019), the findings suggest that developing temporal expectation enables premature accumulation of an evolving perceptual content, leading to a weaker repulsion.
Chapter 5: Study 4―Rapid stimulus alternation
In Study 4, I aimed to quantify the temporal resolution and temporal extent of orientation repulsion in the same paradigm. Temporal resolution and temporal extent are operationally defined as the minimal time to induce repulsion and the maximal time to increase repulsion, respectively. I used an alternating pair of inducer stimuli: one tilted 15° (optimized to induce the maximal repulsion) and the other had a complementary orientation distribution, effectively fused into a concentric pattern with rapid alternation. I quantified repulsion at various alternation frequencies. The constraint that a vertical target was briefly flashed at the midpoint of the duration of the 15° inducer allowed the time window for potential contextual modulation to widen systematically with decreasing frequency.
When an orthogonal pair of D2 patterns, a type of grating whose luminance modulation in a particular orientation was the second-order partial derivative of an isotropic 2D-Gaussian (introduced by Motoyoshi & Nishida, 2001), was used as the inducer, repulsion increased as inducer duration increased from 20 to 30 ms and then leveled off (Experiment 10). When a complementary pair of custom-made textures was used as the inducer, repulsion continued to increase until the duration reached 200 ms (Experiment 11). This gradual increase of repulsion was observed regardless of conscious accessibility to the orientation of the inducer accompanying the target (Experiment 12). I also confirmed that, to quantify the temporal resolution of repulsion, an alternating pair of inducer stimuli must either cause opposing illusions or fuse into a pattern having no orientation bias because repulsion always occurred regardless of inducer orientation if neither condition was met (Experiment 13). These findings reveal that contextual modulation in orientation occurs at a high temporal resolution and continues to a long temporal extent, corroborating my view that perceptual content evolves prior to the awareness of a target in orientation repulsion.
Chapter 6: Study 5―Reverse correlation
In Study 5, I aimed to examine how the dynamic formation process of orientation repulsion depends on spatial distance. Using a psychophysical reverse correlation method, a previous study estimated the temporal kernel of inducing stimuli contributing to repulsion as within 100 ms around the target (Mareschal & Clifford, 2012). Extending this paradigm, I estimated the peak time of the temporal kernel at three different spatial distances between the target and inducing stimuli. Given that modulatory signals from a more distant surround should take more time to be transmitted, the peak time was predicted to shift earlier into the past as the inducer became farther from the target.
However, the result was the opposite. I found that as the spatial distance became longer, the peak time of the temporal kernel shifted into the future relative to the target (Experiment 14). To account for this counterintuitive finding, I suggest that contextual modulation is an active process: it may start with query-like signals sent to the surround after detecting a behaviorally relevant target rather than the surround automatically sending modulatory signals regardless of what stands as the target. This “querying” process inevitably lags behind the direct input, and the difference in the lag at different spatial distances may underlie the temporal evolution of orientation repulsion.
Chapter 7: General discussion
Overall, my dissertation provides novel insights into the dynamic formation process of orientation repulsion. I have demonstrated that repulsion builds up with extending inducer duration (Chapter 5) while querying the broad spatial extent of surrounds about the contextual information (Chapter 6). When this dynamic formation process was prematurely terminated (Chapter 2), facilitated in a bottom-up (Chapter 3) or top-down (Chapter 4) manner, the perceptual content incompletely modulated by spatial contexts arises in the awareness, giving a weaker repulsion.
All the findings consistently suggest that content of orientation repulsion undergoes a gradual temporal evolution, yet observers can consciously access the final output of this process. I consider that the dynamics of contextual modulation for repulsion may be realized by the iso-orientation surround suppression, especially operated in V1 (e.g., DeAngelis et al., 1994). Assuming that the orientation tuning for suppression signals sharpens over time (see Ringach et al., 1997), the tuning for the target will gradually shift against the orientation of inducers. Furthermore, the findings about temporal facilitation effects on repulsion (Chapters 3 and 4) support the view that the content of awareness is constructed as a whole-brain process beyond the visual cortex. I speculate that the premature content may have functional significance in conditions where a speeded reaction is more critical than creating an accurate representation, such as in the conditions of object localization and saliency computation.
The size of repulsion is advantageous among various measures of consciousness because it focuses on definitely suprathreshold content and is immune to response bias. Such a robust measure of contextual phenomena may also contribute to a deeper understanding of symptoms in clinical disorders.
To conclude, contextual modulation underlying the awareness of a target in orientation repulsion is a temporally unfolding process that involves multiple mechanisms with a range of spatiotemporal properties and can be probed with various psychophysical methodologies I introduced in this dissertation.
