• Cheries, Wynn, & Scholl
• Choi & Scholl
• Flombaum, Scholl, & Pylyshyn
• Franconeri, Pylyshyn, & Scholl
• Isola, Turk-Browne, Scholl, & Treat
• Junge, Chun, & Scholl
• New & Scholl
• Scholl
• Shankar, Flombaum, & Scholl
• Turk-Browne & Scholl
• Yi, Turk-Browne, Flombaum, Scholl, & Chun

Making sense of the visual world requires keeping track of objects as the same persisting individuals over time and occlusion. Here we explore a particular aspect of the processes and representations that support this ability in two ways, employing looking-time measures with 10-month-old infants. First, we demonstrate that persisting object representations can be maintained over brief interruptions from additional independent events -- just as your memory of a traffic scene may be maintained through a brief glance in the rearview mirror. In particular, infants maintained accurate representations of the number of dolls in a display while they were behind a screen, despite an interruption wherein an additional novel salient object traversed the display between the initial presentation and test, while the dolls were out of view. Second, we demonstrate that this ability is nevertheless subject to an object-based limit: if the same extrinsic interruption is simply segmented into 4 (or even 2) objects, then it will impair the maintenance of other persisting objects: now the interruption -- despite being perfectly equated in terms of overall salience, duration, and physical extent -- destroys the representation of the number of dolls behind the screen. These experiments demonstrate how object representations can be studied via their 'interruptibility' and the results are consistent with the idea that infants' persisting object representations are constructed and maintained by capacity-limited mid-level 'object-files'.

A prominent trend in recent visual cognition research has been the demonstration of impaired visual awareness without attention. An especially striking example is change blindness, wherein surprisingly salient changes can go undetected when the changed objects or regions are unattended. Most such demonstrations induce inattention to the changing object(s) by (1) using displays with at least several objects (yielding other possible foci for attention), and/or (2) using extrinsic disruptions such as eye movements, occluders, or global transients (to directly distract attention and/or to mask the change). Here we explored a type of change blindness in much simpler events, when observers viewed only two objects, with no extrinsic interruptions, and had to fully attend to those objects -- reporting everything about them immediately after each 1-second trial. The two objects interacted in an ambiguous dynamic pattern, seen to be either 'bouncing' off each other or 'streaming' past each other on opposing linear trajectories. We nevertheless observed considerable change blindness: observers failed to notice when the two objects suddenly swapped colors at various points during their motions. This effect was not due to any baseline visual disruption from the motion, however, since observers readily noticed the sudden introduction of new colors into the display during the same events. These results thus demonstrate a type of change blindness in especially simple displays, and suggest that surface feature information is not reliably bound in memory to specific objects, but is rather tied to events as wholes.

A considerable amount of research has uncovered several heuristics that the visual system employs to keep track of objects through frequent periods of occlusion. Relatively little work, by comparison, has investigated the on-line mechanisms that implement and support these heuristics. We explored how attention is distributed throughout a display when featurally identical objects become momentarily occluded during multiple object tracking (MOT). Observers tracked three targets among three distractors as they moved haphazardly during 10 second trials. All objects periodically became occluded when they passed behind two visible static 'walls'. During tracking, observers also had to detect small probes that appeared sporadically on targets, distractors, occluders, or empty space. Though occlusion did not impair MOT, probe detection rates for these categories confirmed the earlier finding that detection on nontargets was worse than on targets or in empty space and also revealed two novel effects. First, probe detection on an occluder's surface was much greater when either a target or distractor was currently occluded in that location, compared to when no object was behind that occluder. Thus object-based attention can still be active in a display even when the attended object is not visible. Second, and more surprisingly, probe detection was always better when objects were occluded (vs. unoccluded) for both targets and distractors. This attentional high-beams effect indicates that the apparently effortless ability to track through occlusion actually requires the active allocation of additional resources, and the current experiments demonstrate a new way that such effects can be discovered and quantified.

As we move about the world, and objects in the world move relative to us, objects constantly move in and out of view as they are occluded by other objects. How does the visual system maintain attention on objects of interest, given such disruptions? To explore the spatiotemporal cues used to link the pre- and post-occlusion views of objects, we asked observers to track a set of moving objects that frequently passed behind static vertical occluders, as we manipulated each object's exit position. Experiment 1 tested whether linking the two views relies on memory for the object's location. When objects exited occluders higher or lower than expected, tracking performance dropped, suggesting that linking the two object views relies on a location 'marker' at the site of disappearance. In Experiment 2, performance was better when objects exited closer to the initial entry location, rather than their expected extrapolated location, suggesting that the marker is not placed at the extrapolated position. In Experiment 3, tracking performance improved when objects reappeared from occluder centers, compared to at edges, again suggesting that the marker is placed close to the initial point of occlusion. Together, these results suggest that when an object is occluded, the occlusion location is a critical factor in linking pre- and post-occlusion views, but not the extrapolated exit point, or even rudimentary elements of scene structure like the edges of the occluder. This simple trick could underlie much of our perception of persisting objecthood when an object disappears from view.

The visual system has the remarkable ability to extract and exploit subtle statistical regularities in our sensory input. In particular, studies of visual statistical learning (VSL) have revealed rapid and robust learning of temporal statistical relationships in sequences of shapes. However, by using objects that vary only along one dimension (shape) these studies leave open the question of whether VSL extracts relationships between feature values or objects. Here we contrast these two possibilities by studying how VSL operates over compound objects: twelve shapes, each paired with one of twelve colors. Observers watched a six minute familiarization sequence of shapes appearing one at a time, and then completed a series of 2AFC familiarity judgments between two smaller test sequences that varied only in terms of their transitional probabilities from familiarization. Critically, shapes in the test sequences were presented monochromatically for one group of observers, and in their original colors for another group. If the underlying units of VSL are unbound features, then learning should be expressed at test in both groups; in contrast, if VSL is object-based, then learning should be expressed only when the colors are maintained. The results supported this latter possibility: when color was maintained at test, observers chose the test sequence with the higher transitional probability more than 80% of the time. In contrast, when the identical familiarization sequences were later tested monochromatically, observers performed at chance (52%). Thus the underlying units of VSL are objects rather than unbound features.

Over repeated exposure to particular visual search displays, subjects are able to implicitly extract regularities that then make search more efficient -- a phenomenon known as contextual cueing (Chun & Jiang, 1998). Here we explore how the learning involved in contextual cueing is formed, maintained, and updated over training. Implicit learning could be rigid after initial formation, predicting a primacy effect biased toward early evidence. Alternatively, learning could be constantly updated, yielding a recency effect biased toward recent evidence. A third possibility is that the input could be averaged cumulatively, yielding no order effects. During training, subjects were exposed to 18 blocks of (24) predictive context displays, where distractor locations were perfectly correlated with the target location, and also 6 blocks of non-predictive context displays, where the very same contexts were not correlated with particular target locations. Subjects who received the predictive blocks first (followed by the non-predictive blocks) showed robust contextual cueing at a later test. However, no contextual cueing was observed in a different group of subjects who received non-predictive blocks first (followed by the predictive blocks). Since the overall exposure to predictive and non-predictive displays was identical for both groups, the significant difference at test is evidence for a primacy effect in contextual cueing. This links contextual cueing to primacy effects in classical conditioning, high-level causal reasoning (Denis & Ahn, 2001), and temporal visual statistical learning (Catena, Scholl, Isola, & Turk-Browne, under review).

Time can appear to slow down in certain brief real-life events -- e.g. during car accidents or critical moments of athletes' performances. Such time dilation can also be produced to a smaller degree experimentally, e.g. when viewing an oddball stimulus in a sequence of otherwise identical stimuli (Tse et al., 2004, P&P). In this project we explored the spatial distribution of time dilation: Does time expand only for the oddball events themselves, or for all events occurring at that time? Observers viewed a sequence of centrally displayed shapes which appeared one at a time, and had to compare the duration of an intermittently appearing green target to the standard duration of the other randomly-colored nontarget shapes. A peripheral gray distractor disc was presented along with each shape for a shorter, fixed interval. Each distractor disc was either static or underwent a salient ('oddball') transformation, such as expanding in size or orbiting the central shape. With static distractors, targets' durations were accurately perceived. However, targets which were accompanied by oddball distractors were perceived to last considerably longer than their true durations, and in fact could not be distinguished from the objectively longer nontarget duration. Time dilation is thus not tied to oddball stimuli per se, and may be an invariably global experience. Additional experiments addressed whether this result is mediated via a spatial gradient of attentional enhancement, and the degree to which it is augmented by incorporating the target and distractor into a single object.

Multiple-object tracking (MOT) is a unique paradigm for studying attention, in several ways: (1) it requires continuous sustained attention over time rather than brief attentional shifts as in spatial cueing studies; (2) it requires attention to multiple objects rather than focal attention to only a single object at a time; (3) it is an inherently active task, rather than requiring sustained but passive vigilance; (4) it varies attentional load directly rather than doing so via temporal limits (e.g. masking or brief displays), and so allows for manipulations that unfold over time; and (5) it yields relatively large and robust effects, making it ideal for studies that need to distinguish several different levels of performance. Each of these features is relevant to real-world visual processing, and we have thus used MOT to study attention under ecologically important constraints. I will describe and demonstrate such studies, focusing on how MOT has led to new discoveries about both the underlying units of attention (contrasting objects, features, spatial areas, and nonsolid substances), and the nature of visual object persistence (how the visual system identifies objects as the same individuals over time, motion, occlusion, and featural change).

Coherent visual experience requires computations that allow us to perceive objects as persisting individuals through time and space. How is this achieved in a world where objects are regularly occluded by other objects? A key generalization from previous work in vision science and cognitive development is that the underlying rules which control object persistence are spatiotemporal in nature, and do not take objects' surface features into account. We tend to see objects as persisting whenever they traverse spatiotemporally continuous paths through occlusion, for example, even if their colors and shapes change dramatically. Here we report a striking exception to this generalization, based on a surface feature which is rarely considered in studies of persistence -- topological structure. We employed a dynamic change detection paradigm as an implicit measure of persistence (Flombaum & Scholl, 2004, VSS). Subjects viewed displays in which objects oscillated behind occluders on spatiotemporally continuous trajectories. Some items changed topologically (gaining or losing a hole) during occlusion, while others changed shape in other ways equated for overall salience, but preserving topology. Observers were worse at detecting additional color changes across occlusion when they accompanied topological changes, apparently because this forced the construction of a new object representation rather than the updating of a previous representation. This indicates that topological structure is not only especially salient, as has been previously demonstrated (see Chen, 2005, Visual Cognition), but that it also serves to define persistence over time. Thus the view that only spatiotemporal dynamics drive object persistence is full of holes.

A central task of vision is to parse undifferentiated input into discrete objects and groups. Visual statistical learning (VSL) may provide an implicit mechanism for such segmentation via the extraction of covariance between features, parts, and groups. However, because the stimuli in previous VSL studies were identical during training and test, it is unclear what is really being learned: the resulting representations could incorporate all visual details of the learning context, or could be more abstract. We have been exploring such issues using 'transfer' designs in which the stimuli differ between training and test. Persistent VSL across changes along a given dimension indicates that such information is not intrinsic to the resulting representation. Here, we report one of the most extreme possible cases of transfer: from space to time. Observers viewed a seven minute sequence of spatial grids, each containing several adjacent shapes. Unbeknownst to observers, shapes were arranged in fixed spatial configurations that could only be segmented into discrete pairs on the basis of covariance. In a subsequent test, observers performed a target detection task on rapid sequences of shapes presented centrally, one at a time. Targets preceded by their spatial mates were detected more quickly than targets preceded by shapes with which they were not paired during training. Because there was no temporal information during training, and no spatial information during test, these results provide striking evidence for a purely associative component of VSL, and also highlight the incredible flexibility of such learning.

A central task in vision is to represent objects as the same persisting individuals even through visual interruptions such as occlusion. Previous research in several areas of cognitive science has identified a powerful principle in such processing: objects must trace continuous paths through space and time. Here, we report novel fMRI evidence for the neural consequences of spatiotemporally continuous vs. discontinuous motion. We measured fMRI adaptation to reveal whether the fusiform face area treats two faces as the same or different (Grill-Spector et al., 1999; Kourtzi & Kanwisher, 2001). The initial display on each trial contained two vertical columns spanning fixation. One face appeared from behind a column, moved to fixation, turned back, and disappeared behind the original column. Immediately afterwards, a second (same or different) face made a similar movement from either the same column (which would be consistent with its being the same reappearing object) or the other column (which would necessarily be a new object, even if it was featurally identical). We hypothesized that two identical faces from the same column would be treated as the same persisting object, resulting in fMRI adaptation. In contrast, we predicted that two identical faces from different columns would be treated as separate objects, reducing fMRI adaptation. Significant fMRI adaptation occurred only when two identical faces were linked as a single object via spatiotemporal continuity. These results provide a novel demonstration of how spatiotemporal cues to object persistence can influence neural processing of object identity in mid-level visual cortical areas.