WILLIAM L. PALYA
JACKSONVILLE STATE UNIVERSITY
The ability of stimuli correlated with successive periods in a fixed interfood interval to support a response that produced or removed them was examined using pigeons. The degree to which those correlated stimuli elicited directed key pecks was also obtained. Stimuli early in the interval functioned as negative reinforcers, and stimuli late in the interval functioned as positive reinforcers. Stimuli correlated with successively later portions of the second half of the interval supported successively higher rates of elicited pecking and, with the exception of the final stimulus, supported successively higher rates of stimulus production. Stimuli in successively earlier portions of the first half of the interval supported successively higher rates of correlated-stimulus removal. This effect occurred in spite of the addition of a conjoint variable-interval dependency for food. An ogive fit to the mean normalized response distributions resulted in r2s demonstrating that most of the variance in the temporal organization of the behavior was accounted for. The findings were taken to indicate that fixed interfood intervals establish bipolar control.
Key words: autoshaping, observing responses, excitation, inhibition, fixed-interval schedule, conjoint schedule, comparator, key peck, pigeons
A slide/tutorial version of this paper is available here.
The increasing rate of operant behavior across a fixed-interval schedule can be seen as the result of successively shorter delays or stronger contingencies between the successive responses and reinforcement (e.g., Dews, 1970). Alternatively, the fixed-interval "scallop" can be seen as the result of controlling stimuli successively more similar to those in effect at the moment of reinforcement (e.g., Skinner, 1938). Skinner took advantage of the opportunity for theoretical simplification offered by adopting the stimulus-control alternative. The underlying stimulus dimension can be elapsed time or the number of responses occurring in the interval. The lack of precise stimulus control can be explained by noting the difficulty of differentiating those stimuli. Ferster and Skinner (1957, pp. 266-319) pursued this position by asserting that with an external "clock," fixed-interval behavior would exhibit precise stimulus control. They argued that an interval with a perfect clock would come to contain only a single response. However, their research with a continuously varying external stimulus did not produce the predicted perfect stimulus control.
The distribution of behavior within an interfood interval may be conceptualized in a different way. Gibbon and Balsam (1981) presented an analytical perspective within which responding to intermittently presented trial stimuli, followed by food presentation in a Pavlovian design, could be seen. They demonstrated that acquisition speed was a function of the ratio of the overall interfood interval, or cycle, to the trial duration. Palya and Pevey (1987) noted that that ratio was also consistent with the asymptotic response rates to stimuli associated with consecutive portions of the last half of an interfood interval. They demonstrated that stimuli in successively later portions of the second half of an interfood clock controlled successively higher rates. Palya and Bevins (1990) continued this line of research by demonstrating that the increasing rate across the second half of an interfood clock occurred in both operant and Pavlovian procedures, as well as under an omission dependency.
These findings, obtained with interfood intervals with explicitly unpaired clock stimuli (i.e., in the omission procedure), make it difficult to argue that the responding in the second half of unsignaled interfood intervals is most likely the result of temporal confusion, as suggested by Skinner's original view. Rather, the distribution of responding to the latter half of an interfood clock may best be taken to indicate that that is the behavioral equilibrium established by the fixed interfood interval itself (cf. Dews, 1970; Palya, 1985; Platt, 1979).
The literature suggests an analogous but mirror-image effect in the first half of the interval. That is, successively earlier portions of an interfood interval are increasingly aversive or negative. Dinsmoor, Lee, and Brown (1986) found that pigeons would work to terminate stimuli associated with the earliest portion of a variable interfood interval. These findings are counter to theoretical perspectives suggesting that earlier portions of the interval should be more reinforcing (Egger & Miller, 1962, 1963) or irrelevant (Shull, 1979).
Obviously, there are a number of views as to how to characterize behavior in an interfood interval most appropriately ( Dinsmoor et al., 1986; Egger & Miller, 1962,1963; Shull, 1979; Staddon & Simmelhag, 1971; Zeiler, 1977). The hypothesis underlying the present research is that the asymptotic behavior to be expected in an interfood interval implementing either operant or Pavlovian contingencies can be broken into roughly two halves. Successively later portions of the second half of an interfood interval should control higher rates of approach, of elicited appetitive behavior, and reinforcing effectiveness. This finding can be described as an increasingly positive asymptotic limit (e.g., Rescorla & Wagner, 1972). In addition, it is hypothesized that earlier portions of the first half of an interfood interval should control higher rates of avoidance, inhibitory control, and negative reinforcing effectiveness. These data are taken to be indicative of a maximally negative asymptotic limit at the point maximally separated from impending food presentation. This negative limit is thought to decrease as time in the interval elapses. The approximate middle portion of the interval is, therefore, thought to be neutral.
The specific aim of the present research was to describe the degree to which the stimuli of an interfood clock can come to function as a positive reinforcer, a negative reinforcer, and an elicitor of directed pecking. This provided a continuous within-bird, within-interval assessment of both the operant and Pavlovian control supported by each portion of an interfood interval. It was accomplished by implementing what was, in effect, an observing response procedure. The extent to which each stimulus supported a response that produced it indicated the degree to which that stimulus functioned as a positive reinforcer. The extent to which each stimulus supported a response that removed it indicated the degree to which it functioned as a negative reinforcer. The rate of key pecking to each clock stimulus itself defined the ability of that stimulus to elicit directed pecking.
Other theoretical perspectives make other predictions. Earlier information can be more reinforcing than subsequent information (Egger & Miller, 1962, 1963). If that were the case, then early clock stimuli should be more positively reinforcing than subsequent stimuli. On the other hand, early portions of a fixed interfood interval can be considered neutral (Shull, 1979). If that were the case, then the early stimuli should function as neither positive nor negative reinforcers.
Six adult experimentally naive White Carneau pigeons, obtained from a local supplier, were used. They were housed under a 19:5 hr light/dark cycle in individual cages with free access to water. All were maintained with pelletized laying mash. As determined by each bird's presession body weight, the number of reinforcers in a session was adjusted from its typical value of 50, so that each bird would be at 80% of its free-feeding weight at the beginning of the next session. Birds very rarely varied by more than a few percentage points from their prescribed weights.
Four experimental chambers were used. The interior of each was a 30-cm cube painted white. A stimulus panel served as one wall of the chamber. It had a feeder aperture 5 cm in diameter medially located 8 cm above the grid floor. Three symmetrically positioned response keys, 2 cm in diameter, were located 9 cm apart and 19 cm above the grid floor and required approximately 15 g (0.15 N) to operate. The translucent Plexiglas keys could be transilluminated by stimulus projectors containing color filters. The filters were the following Rosco theatrical gels: pink (829), red (26), orange (22), amber (615), yellow (12), green (874), turquoise (877), blue (657), and purple (843) . A Lee color correcting filter (218) was used to produce white. Two houselights were located 28 cm above the grid floor and 9 cm apart. Ventilation was provided by an exhaust fan mounted on the outside of the chamber. A white-noise source provided ambient masking noise. The houselights and keylights were off and the magazine light was on during food presentation. Stimulus events were controlled and key pecks were recorded by a computer system (Walter & Palya, 1984).
All procedures were variations of a clocked fixed-time (CFT) schedule ( Palya, 1985). The core procedure consisted of a fixed interfood interval segmented into 10 time periods, each potentially designated by a different key color on the center key. This clock key was dark when a clock stimulus was not in effect. A different color sequence was used for each of the 6 birds. A 3-s food presentation occurred at the end of every 60-s trial. Responses to the observing-response (obs) key temporarily illuminated the clock key with the stimulus appropriate for that temporal position in the interval. Responses to the negative observing response (nobs) key temporarily terminated any clock stimulus in effect.
Pecking a particular side key was considered adequately supported and reliable when 25% of the trials contained an obs response when the clock key was dark or a nobs response when the clock key was illuminated. The distribution of responding to the obs key was used to index the positively reinforcing effectiveness of the stimuli correlated with each portion of the interval. The distribution of responding to the nobs key was used to index the negative reinforcing effectiveness of the stimuli correlated with each portion of the interval. The distribution of responding on the illuminated center clock key provided an index of the eliciting effectiveness of each stimulus.
A basic trial is illustrated in Figure 1. At the beginning of each session, the early stimuli of the clock (presumed to be aversive) were presented without requiring an observing response. A peck to the nobs key turned these stimuli off, and any subsequent obs responses turned them back on. The final stimuli of the clock (presumed to be reinforcing) were not presented unless an observing response occurred. A peck to the obs key was necessary to turn these stimuli on, and any subsequent nobs responses turned. them back off. The effects of side-key pecks during the third and eighth stimulus times are illustrated in Figure 1.
In order to quantify more adequately the point in the interval at which the clock stimuli changed from aversive to neutral and from neutral to reinforcing, a titrating schedule was used. The schedule behaviorally assessed the value of the clock stimuli on each trial, and subsequently presented putative aversive stimuli without a response requirement (and the bird could turn them off briefly). The schedule withheld the reinforcing stimuli that the bird could turn on briefly. The titration procedure adjusted the changeover between "free" stimuli and response-dependent stimuli to the stimulus following the temporal position of the last nobs response and to the stimulus preceding the temporal position of the first obs response.
In order to identify stimuli that were both (or neither) positively and negatively reinforcing, it was necessary to use every other trial to determine the earliest stimulus with a positive reinforcing effect (obs titration point) and the alternate trials to determine the last stimulus with a negative reinforcing effect (nobs titration point). The only difference in the trial types was the titration point and the titration algorithm that was affected by the behavior. Clock stimuli could be turned on or off at any time throughout the interfood interval during either type of trial. At the beginning of each session, the changeover from normally presented clock stimuli to clock stimuli that were normally off preceded Stimulus 5 on the first nobs test trial and followed Stimulus 5 on the first obs test trial. The titrating schedule adjusted the changeover points based on the recent behavior on the side keys. If a bird never responded to the obs key, the obs point was decremented by one stimulus each obs test trial to the beginning of the interfood interval (all stimuli were off and had to be turned on by the bird); if a bird never responded to the nobs key, the nobs point was incremented by one stimulus each nobs test trial to the end of the interfood interval (all stimuli were on and had to be turned off by the bird).
Subjects and apparatus. Six naive pigeons and four chambers were used.
Procedure. Following magazine training, the birds were exposed to pretraining based on an interfood clock procedure for 27 sessions of approximately 50 trials each. In this procedure a sequence of 10 6-s keylights was followed with food presentation, irrespective of behavior. This sequence was immediately repeated with no intertrial interval. The consecutive stimuli appeared on the same key, but following each food presentation the next trial was started on a randomly selected key. For half of the birds the other two (nonclock) keys were dark, and for the remaining half of the birds the alternative keys were illuminated white.
Following the pretraining conditions, the test phases were implemented. In Phase 1, the birds were exposed to 30 sessions with the basic titrating obs/nobs procedure in the effect. The center key was designated as the clock key. The birds that had white alternate keys now also had an obs contingency on the right key and a nobs contingency on the left key. The birds that had dark alternate keys were now exposed to a magenta obs key and an amber nobs key. The left/right position of these hues and their correlated functions were randomly selected for each trial. For all birds, an obs response or a nobs response changed the state of the clock key for 2 s.
The second phase simply extended the duration of the consequence of an obs response to the end of the trial. As a result, a single obs response would instate the clock until the end of the trial or until a nobs response occurred. A single nobs response continued to turn off the clock for 2 s or until an obs response occurred. All other aspects of this procedure were identical to the basic titrating obs/nobs procedure. This phase was in effect for 20 sessions.
Results and Discussion
Although the rates to the clock key and the behavior maintained by the onset and offset of the clock stimuli were in the expected direction, the effects were neither large nor universal. However, the number of birds that acquired consistent responding to a side key and the consistency with which that behavior was distributed indicated that an interfood interval can, under some circumstances, come to establish increasingly positive stimuli in the latter half of the interfood interval and increasingly negative stimuli in the successively earlier portions of an interfood interval.
The distribution of responding for each of the 6 birds under each of the procedures is depicted in Figure 2. The upper two panels illustrate the effective obs responses (pecks to the obs key when the clock was off). The center two panels illustrate the behavior directed to the clock key when the clock key was on (elicited pecks), and the bottom two panels illustrate the effective nobs responses (pecks to the nobs key when the clock was on). The left set of frames illustrate the results of Phase 1 when the obs consequence duration was 2 s. The right set of frames illustrates the results when the consequence of an obs key peck was in effect for the remainder of the trial ("rest"). As can be seen in the birds that responded, the obs distributions show virtually zero rates until the midpoint, followed by successively increasing, or increasing then decreasing, rates to the end of the interval. Even though the absolute nobs rates were substantially lower, the nobs distributions show the inverse effect. Nobs responding showed an earlier onset and was typically lower in the second half of the interval. Turning off the clock was generally the first side-key behavior to occur, and turning on the clock was generally the last side-key behavior to occur. The rates to the clock stimuli themselves generally mirrored, but were higher than, the rates maintained by the onset of the clock stimuli. A comparison of the obs rates in Phases 1 and 2 indicates a rate decrement when only a single key peck was necessary to instate the clock for the remainder of the trial.
These data do not make clear what specific procedures ensure between-subject generality in the acquisition of behavior maintained by the presentation or removal of clock stimuli. It is clear, however, that if the behavior is acquired, then its temporal distribution is under the control of position in the interfood interval.
The opportunity for simple interpretations that emerge from simple procedures encourages designs that specify only the contingency of interest and that allow unconstrained behavior to come into contact with that contingency. That was the approach taken, but unfortunately, this path was not sufficiently effective. This experiment demonstrated that simple exposure to the various procedures was not sufficient to establish or maintain the behavior followed by the onset or offset of the clock stimuli in all birds. The failure to establish substantial obs and nobs responding in most of the birds may have been due to their initially low operant level to the side keys, despite pretraining experience of pecking these keys. It seems reasonable to assume that the birds had had sufficient experience with the correlation of clock stimuli and position in the interval and with the consequences of responding to the side keys. However, that experience was, in fact, insufficient for the programmed contingencies to come to control the behavior reliably.
Some contingencies and reinforcers may require more exposure or exposure of a type that many subjects may not produce without some explicit contingency in addition to the one of interest. The strategy taken in Experiment 2 was to seek an "establishing" schedule that was sufficient to establish, as well as maintain, behavior in all subjects with simply the onset or offset of the clock stimuli without explicit shaping of that behavior with primary reinforcers. Such a schedule would be useful in designs, such as the present one, in which the factors that establish the behavior are seen only as precursors, whereas the temporal distribution of the control exerted by those stimuli is the primary focus. Two likely contributing factors to the bird-to-bird variability in Experiment 1 were (a) the degree to which the birds "attended to" or were actually exposed to the stimuli of the clock, and (b) the degree to which the birds engaged in unrecorded avoidance, such as simply turning away from the key. The establishing schedules in Experiment 2 were implemented to explore these possibilities.
The procedures of Experiment 2 assessed the ability of the clock stimuli to generate and maintain the behavior that produced or removed them, when pecking to the clock key itself was well maintained throughout the interfood interval with a conjoint variable-interval (VI) schedule. It was expected that this procedure would decrease the relative reinforcement of simply turning away from the key during the beginning of the interval and would thereby also assure exposure to the stimuli early in the interval.
Subjects and apparatus. The 6 pigeons and four chambers from Experiment 1 were used.
Procedure. The pigeons were initially exposed to the basic titrating obs/nobs procedure with an added conjoint VI l-min response requirement on the clock key in order to maintain key-directed behavior throughout the interfood interval. In addition to the response independent food presentation scheduled to occur at the end of each 60-s trial, pecks to the clock key were reinforced intermittently according to a VI l-min schedule. During the VI reinforcement operation, the clock time did not elapse. During the fixed-time reinforcement operation, the VI time did not elapse. The consequence of an obs peck changed the clock for the remainder of the trial, or until a nobs reversed the state of the clock. Nobs responses turned the clock off for 2 s. This first phase was in effect for 20 sessions. The second phase, which also lasted 20 sessions, decreased the duration of the conjoint VI reinforcer from 3 s to 2 s in order to reduce the relative value and presumably the saliency of the VI reinforcement schedule with respect to the fixed-time reinforcement schedule. All other aspects of the procedure were the same. The third phase, which was in effect for 2Q sessions, reduced the duration of the stimulus consequence for an obs response to 2 s; otherwise, the procedures were unchanged. The next phase attempted to reduce the relative value of the conjoint VI schedule further by decreasing the reinforcement rate to a VI 2-min schedule. This phase (Phase 4) lasted 20 sessions. Phase 4 was followed by a return to the Phase 1 baseline procedure of Experiment 1 by removing the conjoint VI schedule on the clock key. This final phase was in effect for 30 sessions.
Results and Discussion
Substantially higher response rates were maintained by the onset and offset of the various clock stimuli when pecking to the clock key was maintained throughout the interval by a conjoint schedule. However, the temporal distribution of side-key behavior controlled by the various side-key contingencies was essentially the same as that obtained in the birds that had reliably responded to the side keys in Experiment 1. Figure 3 illustrates the effective obs and nobs behavior for each of the 6 birds in each of the five procedures of this experiment. The five distributions for each bird are presented together in the same frame, because the procedures had only a marginal and inconsistent effect on the temporal distribution of the obtained behavior. The number of effective obs pecks per 6-s bin is indicated with y values above the x axis, and the number of effective nobs pecks per 6-s bin is depicted with y values below the x axis. Rates maintained by the offset of the clock stimuli were generally highest near the beginning of the clocked interfood interval and decreased after the approximate midpoint. The notable exceptions were Bird 313, which also showed a rate increase at the very end of the trial, and Bird 360, which exhibited very low nobs rates. The rates maintained by the onset of the clock stimuli were generally low until before the approximate midpoint and increased to the end of the clocked interfood interval. The exceptions were Bird 352, which began responding earlier in the interval during some procedures, and Bird 400, which did not reliably peck the obs key. Not surprisingly, the magnitude of the rate increase across the interval for the obs responses was reduced when a single obs response instated the clock for the remainder of the trial. This procedural difference produced the flatter obs functions seen in each frame.
The four frames of Figure 4 illustrate details of the behavior within the interfood interval for Bird 357. Time in the interfood interval is depicted across the x axis from one fixed-time reinforcer to the next; the fixed-time (above the x axis), as well as any conjoint (below the y axis), reinforcers are indicated with an asterisk. The positive and negative y axis is used to represent several events. The stippled area denotes the state of the clock; above the x axis indicates clock stimulus on, and below the axis indicates clock stimulus off. Long tick marks indicate obs and nobs responses; they are drawn above or below the axis, respectively. Responses to the clock stimulus key are indicated by short tick marks straddling the x axis. Multiple closely-spaced responses appear as thicker tick marks. A clock state change that occurred as the result of an obs or nobs response is preceded by that type of tick mark. Two-second obs or nobs events of programmed duration are seen as consistent-width stippled bars. If side-key behavior terminated a previously instated 2-s clock state, it truncates the width of the preceding stippled bar on the event diagram. Clock state changes that were programmed by the consequence timer or the titration algorithm simply cross the axis without an associated initial side-key response. The point in the fixed interfood interval at which the titration algorithm changed the schedule from normally off clock stimuli to normally on clock stimuli is indicated with an arrow.
The upper frame illustrates the behavior in the basic clocked fixed-time schedule with an added conjoint contingency (Phase 1, Session 20, Trial 19). An obs response instated the clock for the remainder of the trial. This bird pecked both side keys on a substantial portion of the trials and occasionally reversed the state of a consequence it had just produced. As can be seen, the trial started with the clock on; then, two ineffective obs responses and a peck to the clock stimulus key occurred. This was followed by two nobs responses, the first of which turned off the clock. This was followed by two additional clock key responses and another nobs response, after which the 2-s consequence timer elapsed and the clock was illuminated again. A clock key response occurred followed by two more nobs responses. Similar nobs behavior continued until the seventh nobs consequence, which was terminated by an obs response. The obs consequence was in turn terminated by a nobs response. This was followed by a clock key response that produced a reinforcer on the conjoint schedule. Subsequently, there was a return of the clock stimulus following the 2-s nobs consequence duration. Following three more nobs responses and an obs response, another food presentation obtained by meeting the VI dependency occurred, this time while the clock was on. The rate to the clock key began to show acceleration that was interrupted by the nobs point (change from normally on to normally off clock stimuli) scheduled by the titrating algorithm. An immediate obs response again instated the clock and clock key responding continued until the fixed-time food presentation.
The second frame illustrates the behavior of this bird when the consequence duration was 2 s for both obs and nobs responses (Phase 3, Session 20, Trial 2). The trial started with the clock on, and a conjoint reinforcer was collected almost immediately; this was followed by further pecking on the clock key. A series of nobs responses were emitted, each turning off the clock for 2 s. Two closely spaced, ineffective obs responses occurred followed by the obs point, after which the clock was in the normally off state. This was followed by a series of obs responses and accelerated clock stimulus key responding.
The third frame illustrates the earlier behavior of this bird under the baseline conditions of Experiment 1, Phase 1 (Session 30, Trial 44) for comparison. This trial was started with the clock on, and no nobs responses occurred. Some responses to the clock stimulus key eventually occurred, followed by the obs point, at which time the default state of the clock reversed. This was followed by obs responses and pecks to the clock stimulus until food presentation. The final frame illustrates the behavior obtained under the same procedure when this bird was returned to baseline in Phase 5 (Session 30, Trial 34). The event log shows two initial nobs responses that resulted in consequences, followed by obs and clock stimulus key responding late in the interval. In this case, food presentation occurred in the presence of a dark clock key.
The trials illustrated in Figure 4 were chosen to provide a detailed description of the behavior typically controlled by these schedules. This figure, therefore, provides perspective to the figures that present only 5-day means of observing response, negative observing response, and clock rates. The Phase 1 record shows a substantial nobs rate when a single obs instated the clock for the remainder of the trial. Reversing the state of the clock key generally occurred on 25% or more of the trials in birds whose behavior was reliably controlled by both the onset and offset of the clock stimuli. The second frame (Phase 3) shows a decrease in the nobs rate and an increase in the obs rate when the obs response consequence was reduced to 2 s. This bird's original exposure to the baseline titrating obs/nobs procedure (illustrated in the third frame) controlled nobs responses on 22% of the trials, and the Phase 5 reinstitution of baseline (bottom frame) controlled nobs responses on 33% of the trials.
As can be seen by comparing the first two frames with the last two frames of Figure 4, the conjoint VI schedule on the clock key increased the overall response rate to the clock key by maintaining behavior to that key throughout the fixed interfood interval. When the clock key was dark, response rates to the clock key were relatively constant. However, when the clock stimuli were on, response rates on the clock key typically accelerated across the latter half of the clocked interfood interval. Rather than abolishing the pattern of behavior under the fixed interfood interval, the VI reinforcers simply added a constant rate enhancement to what remained a fixed 60-s temporal interval when the clock was in effect.
Therefore, the most obvious interpretation for why the conjoint schedule so strongly influenced the side-key behavior is that it maintained behavior to the clock key throughout the interfood interval with a primary reinforcement contingency. As a result, terminating exposure to the early clock stimuli by simply turning away from the key early in the interval was associated with a lower reinforcement rate. Terminating exposure to the early clock stimuli by pecking the nobs key was more reinforcing because it removed the clock stimulus without decreasing the reinforcement rate.
It is interesting to note that the differential effects of the various clock stimuli were established by their consistent and differential relationship with the temporal imminence of the scheduled fixed-time reinforcement, rather than by their equal probabilistic relationship with the conjoint VI reinforcers. As a result, the clock did not simply vary from a "negative" absence of food to a "positive" imminence of food, but rather varied from a "negative" moderate probability plus remoteness of food to a "positive" moderate probability of food plus the imminence of food. This strong control by the fixed interfood clock stimuli is consistent with that demonstrated by Palya and Bevins (1990). They found that the control exerted by the successive clock stimuli was preserved even when the interfood interval began at randomly selected stimuli in the clock. Under that procedure, the interfood interval was variable.
Figure 5 illustrates the trial-to-trial changes in the obs and the nobs titration points for Bird 352 with increasing experience. It depicts the behavior during Phase 5 when the baseline titrating obs/nobs procedure without the conjoint schedule was reinstated. Several interesting effects are illustrated. Bird 352 showed a shift in the temporal distribution of obs and nobs behavior following reinstatement, and it emitted both obs and nobs behavior during a portion of the interfood interval.
Figure 5 shows the obs and nobs titration points. The functions indicate the point in each consecutive obs and in each consecutive nobs test trial at which the clock was changed from normally on to normally off. These are the behaviorally established crossover points from aversive-or-neutral to reinforcing clock stimuli and from aversive to neutral-or-reinforcing clock stimuli. With no responding to the side keys, the obs point was titrated to the trial onset (x axis), and the nobs point was titrated to the end of the trial (its maximum).
Figure 5 demonstrates that when the conjoint schedule was removed, the obs point for Bird 352 shifted to later in the interval (from approximately Segment 5 to Segment 7) and reliable nobs responding came under the control of the interfood interval. The reacquisition of nobs behavior in this phase was indicated by the emergence of unsystematic oscillations around Segment 7 following frequent titration to the interval end point at the beginning of the phase. Taken together, these two frames show that eventually clock stimuli after Segment 6 or 7 were positively reinforcing and stimuli preceding Segment 6 or 7 were negatively reinforcing for this bird, and that those changeover points were consistent across test trials.
Figure 6 provides an aggregate quantification of the temporal distribution of side-key behavior across the interfood interval. The left column of frames presents the obs and nobs rates for the procedure with an added conjoint VI schedule. The right column of frames presents the obs and nobs rates during the return to baseline.
The upper two frames in each column present the data for each bird in terms of pecks per bin. The middle frames replot these data as normalized functions (Dews, 1970; Gibbon, 1991). Each bird's obs (nobs) pecks per bin across the fixed interfood interval was specified as a proportion of the pecks in the bin with the maximum number of obs (nobs) pecks for that bird. It should be noted that these frames quantify the temporal properties of the distribution at the expense of portraying the absolute rate. As can be seen by examining the data in the frames immediately above these frames, the nobs rates usually were lower than the obs rates. The data points in the bottom frames with fitted functions indicate the mean relative number of pecks per bin in each 10th of the interfood interval for both obs and nobs behavior combined, combined across birds. The mean normalized nobs pecks were subtracted from the mean normalized obs pecks. To the extent that the bird-to-bird variation in the relative distribution of behavior across the interfood interval was random, this operation provided the best predictor of the shape of the distribution of obs and nobs behavior across an interfood interval.
As can be seen in the upper frame of each column of Figure 6, obs rates were highest in the final or penultimate bin of the interval, whereas nobs rates were highest early in the interval (with the exception of the function for Bird 352 under the CFT-only procedure). Even though the similarities in the temporal distribution of responding across the interfood interval under the two procedures outweigh their differences, there were differences that appear reliable. The nobs rates were higher overall, and the obs rates were higher in the early portion of the interval under the CFT plus conjoint schedule.
The bottom frame in each column provides an ogival description of the shape of these distributions. The ogive fits of the relative distribution of behavior across the interfood interval produced r2s of 0.97 and 0.94. (The r2s for straight lines were 0.88 and 0.76, respectively.) The ogive and straight-line fits for the relative distributions, including bird-to-bird variability, produced r2s of 0.26 and 0.24 for the CFT plus conjoint schedule and 0.24 and 0.20 for the CFT alone.
Stimuli correlated with the successive portions of the latter part of an interfood interval were for the most part increasingly positive, whereas stimuli correlated with successively earlier portions of an interfood interval were increasingly negative. Intervening stimuli were intermediate. Explanations of responding based solely on the decay of a positive process at successively earlier times before the reinforcer (e.g., Ferster & Skinner, 1957, pp. 813) cannot handle the finding that early stimuli supported their removal. However, a view suggesting that a food-to-food temporal gradient establishes both an initial negative component and a subsequent positive component is able to do so. A bipolar view of the behavior to be expected across an interfood interval is consistent with the literature; variants of this idea have been proposed previously ( Segal, 1962; Staddon, 1983).
Figure 7 provides a spatial representation of the relative strength and vector of the behavior predicted by this bipolar view. It indicates the output to be expected both within interfood intervals (x axis) and across trials with increasing experience (z axis). This general perspective is consistent with the behavior change predicted to occur by linear-operator models (e.g., Rescorla & Wagner, 1972). The bipolar perspective views the typical learning curve as only a single element (the rightmost edge of the surface) in an entire family of growth functions (changes on z axis) specifying the behavior that results from a repetitive exposure to an interfood interval. This surface could also be seen as a continuous, bipolar extension of the output predicted by a comparator view (changes on x axis) (e.g., Gibbon & Balsam, 1981). The bipolar perspective would see a typical intertrial interval followed by a trial stimulus as a special case that dichotomizes the bipolar function at the onset of the trial stimulus. The task of future research is to provide a functional analysis of the factors that determine the crossover point, or bias, and the relative amplitude of the negative and positive components.
The present results, demonstrating a bipolar behavioral gradient established by an interfood interval, may provide for a more systematic treatment of several frequently invoked explanatory processes. Delay of reinforcement, higher order conditioning, chaining, and information are all used to explain how reinforcement is effective in situations that involve other than strictly simultaneous pairings of reinforcers and initially neutral stimuli. Typically, some opposing process (such as generalization decrement) is also invoked to account for the loss of the reinforcing effectiveness with increasing temporal separation. Unfortunately, without a systematic framework, these opposing processes are easy to invoke arbitrarily to whatever extent is necessary to account for the data. The present findings suggest that the spread of effect may be described by an ogival function between the point maximally separated from the impending reinforcer, or Smin, and the point immediately preceding reinforcement, or Smax Approach or terminal behavior (cf. Staddon & Simmelhag, 1971) should be expected to extend back to the point at which the function becomes negative.
The general contextual framework within which the bipolar effect could best be placed is open to speculation. It could be argued that a bipolar gradient across an interfood interval had adaptive significance in the evolutionary time scale, or that the similar bipolar properties of lateral inhibition have been shown to be adaptive in perceptual systems. A third alternative seems practical, in that it provides for a functional analysis. The bipolar view can be integrated within a framework of other behavioral processes. This line of reasoning argues that the context established in an interfood interval could be seen as a continuous choice paradigm with systematically changing, inversely related schedules ( Herrnstein, 1970). Immediately before food presentation, the terminal behavior is most strongly evoked or reinforced, whereas escape or interim behavior is only weakly evoked or reinforced. Earlier, at the onset of the interfood interval, the reverse is true. The period following the approximate midpoint of the interval would be the point at which the two schedules are roughly equal. Control would be expected to be weakest at this point. This prediction is consistent with the finding that "facultative behavior" is most likely in the middle of the interval (Staddon, 1977).
Dews, P. B. (1970). The theory of fixed-interval responding. In W. N. Schoenfeld (Ed.), The theory of reinforcement schedules (pp. 43-61). New York: Appleton-Century-Crofts.
Dinsmoor, J. A., Lee, D. M., & Brown, M. M. (1986). Escape from serial stimuli leading to food. Journal of the Experimental Analysis of Behavior, 46, 259-279.
Egger, M. D., & Miller, N. E. (1962). Secondary reinforcement in rats as a function of information value and reliability of the stimulus. Journal of Experimental Psychology, 64, 97-104.
Egger, M. D., & Miller, N. E. (1963). When is a reward reinforcing? An experimental study of the information hypothesis. Journal of Comparative and Physiological Psychology, 56, 132-137.
Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. New York: Appleton-Century-Crofts.
Gibbon, J. (1991). Origins of scalar timing. Learning and Motivation, 22, 3-38.
Gibbon, J., & Balsam, P. (1981). Spreading association in time. In C. M. Locurto, H. S. Terrace, & J. Gibbon (Eds.), Autoshaping and conditioning theory (pp. 219-253). New York: Academic Press.
Herrnstein, R. J. (1970). On the law of effect. Journal of the Experimental Analysis of Behavior, 13, 243-266.
Palya, W. L. (1985). Sign-tracking with an interfood clock. Journal of the Experimental Analysis of Behavior, 43, 321-330.
Palya, W. L., & Bevins, R. A. (1990). Serial conditioning as a function of stimulus, response, and temporal dependencies. Journal of the Experimental Analysis of Behavior, 53, 65-85.
Palya, W. L., & Pevey, M. L. (1987). Serial conditioning as a function of parametric variations of an interfood clock. Animal Learning & Behavior, 15, 249-262.
Platt, J. R. (1979). Temporal differentiation and the psychophysics of time. In M. D. Zeiler & P. Harzem (Eds.), Reinforcement and the organization of behaviour (pp. 1-29). New York: Wiley.
Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 64-99). New York: Appleton-Century-Crofts.
Segal, E. F. (1962). Exteroceptive control of fixed-interval responding. Journal of the Experimental Analysis of Behavior, 5, 49-57.
Shull, R. L. (1979). The postreinforcement pause: Some implications for the correlational law of effect. In M. D. Zeiler & P. Harzem (Eds.), Reinforcement and the organisation of behaviour (pp. 193-221). New York: Wiley.
Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. New York: Appleton-Century-Crofts .
Staddon, J. E. R. (1977). Schedule-induced behavior. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of operant behavior (pp. 125-152). Englewood Cliffs, NJ: Prentice-Hall.
Staddon, J. E. R. (1983). Adaptive behavior and learning. Cambridge: Cambridge University Press.
Staddon, J. E. R., & Simmelhag, V. L. (1971). The "superstition" experiment: A re-examination of its implications for the principles of adaptive behavior. Psychological Review, 78, 3-43.
Walter, D. E., & Palya, W. L. (1984). An inexpensive experiment controller for stand-alone applications or distributed processing networks. Behavior Research Methods, Instruments, & Computers, 16, 125-134.
Zeiler, M. (1977). Schedules of reinforcement: The controlling variables. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of operant behavior (pp. 201-232). Englewood Cliffs, NJ: Prentice-Hall.
Received November 9, 1992 Final acceptance March 17, 1993
This research was supported by NSF Grant BNS 8808409 to William L. Palya. The author gratefully acknowledges the contributions of Donald Walter for data analysis and discussions, Keith Easterling for discussions, Helen Bush and Rick Bevins for meticulously conducting sessions, Josey Chu and Sheila Katayama for artwork, and Elizabeth Palya for contributions in all phases of this research. Correspondence and requests for reprints should be sent to William L. Palya, Department of Psychology, Jacksonville State University, Jacksonville, Alabama 36265 ( .