The ventral attentional network (VAN) is thought to drive “stimulus driven attention” [e.g., Asplund, C. L., Todd, J. J., Snyder, A. P., & Marois, R. A central role for the lateral prefrontal cortex in goal-directed and stimulus-driven attention. Nature Neuroscience, 13, 507–512, 2010; Shulman, G. L., McAvoy, M. P., Cowan, M. C., Astafiev, S. V., Tansy, A. P., D' Avossa, G., et al. Quantitative analysis of attention and detection signals during visual search. Journal of Neurophysiology, 90, 3384–3397, 2003]; in other words, it instantiates within the current stimulus environment the top–down attentional biases maintained by the dorsal attention network [e.g., Kincade, J. M., Abrams, R. A., Astafiev, S. V., Shulman, G. L., & Corbetta, M. An event-related functional magnetic resonance imaging study of voluntary and stimulus-driven orienting of attention. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 25, 4593–4604, 2005]. Previous work has shown that the dorsal attentional network is sensitive to trial history, such that it is challenged by changes in task goals and facilitated by repetition thereof [e.g., Kristjánsson, A., Vuilleumier, P., Schwartz, S., Macaluso, E., & Driver, J. Neural basis for priming of pop-out during visual search revealed with fMRI. Cerebral Cortex, 17, 1612–1624, 2007]. Here, we investigate whether the VAN also preserves information across trials such that it is challenged when previously rejected stimuli become task relevant. We used fMRI to investigate the sensitivity of the ventral attentional system to prior history effects as measured by the distractor preview effect. This behavioral phenomenon reflects a bias against stimuli that have historically not supported task performance. We found regions traditionally considered to be part of the VAN (right middle frontal gyrus, inferior frontal gyrus and right supramarginal gyrus) [Shulman, G. L., McAvoy, M. P., Cowan, M. C., Astafiev, S. V., Tansy, A. P., D' Avossa, G., et al. Quantitative analysis of attention and detection signals during visual search. Journal of Neurophysiology, 90, 3384–3397, 2003] to be more active when task-relevant stimuli had not supported task performance in a previous trial than when they had. Investigations of the ventral visual system suggest that this effect is more reliably driven by trial history preserved within the VAN than that preserved within the visual system per se. We conclude that VAN maintains its interactions with top–down stimulus biases and bottom–up stimulation across time, allowing previous experience with the stimulus environment to influence attentional biases under current circumstances.