Many years ago when I was a graduate student, Mike Wade suggested that we need to consider two distinct types of population structure, mating structure and interaction structure. At the time I was quite naïve and I constructed my own meaning around that idea. I doubt he was thinking about it in the way I will talk about today. Mike suggested is that for any population we really need to think about the mating structure and the interaction structure. The mating structure being the range over which an individual mates, and the probability of mating with different individuals. For interaction structure I believe Mike was thinking about indirect genetic effects and how non-random interactions would influence traits.
I want to suggest that the “interaction structure” is actually much more complicated than that. First, there is the interaction structure, which is my vision of what Mike was talking about: what does an individual interact with, and how does it affect their phenotype. Second, when we are talking about heritability we are talking about intergeneration correlations. This becomes particularly obvious with higher levels of selection, where the patterns of co-migration can qualitatively change the heritability of a trait. Finally, when we are talking about evolution by natural selection we speak of selection among one object and within another object. For example selection may be among individuals and within populations, or among cells within the organism, or among populations within a metapopulation.
There is no reason that any of these match up. For example, in a plant the mating structure may be determined by pollinator flight patterns, the interaction structure might be at the level of the local neighborhood and much smaller than the pollen flow distance, seed flow distance might be the determinant of comigration and the heritability of contextual traits, and the selection structure is determined by the behavior of the herbivore that is deciding where and what to graze.
This is something that is rarely thought about, but it actually can have profound influences on evolution. Consider the interaction of two of these structures, mating structure and selection structure. In nearly every selection experiment I have ever seen these two are set to be the same. Thus, in a typical Drosophila selection experiment all of the flied in a particular bottle will be subjected to the same selection regime. Thus, the mating structure is the bottle (although it may not be random mating within that bottle), and the selection structure the “within” is also the bottle. How might you change that? Well, you could have a set of bottles, some with, say, an insecticide, and some without. Selection is now taking place at the level of the bottle. If the bottles were mixed and redistributed every generation then the mating structure would be taking place over a set of bottles (a metabottle?). Thus, the selection structure would be smaller than the mating structure. Conversely, you could have a set of bottles, and choose the most resistant flies from the entire set of bottles, and then use a migration scheme between bottles in which flies preferentially went back to their own bottle, but when their home bottle was full they would migrate to less successful bottles. In this case the mating structure would be smaller than the selection structure.
Considering the first scenario in which the selection structure is smaller than the mating structure, this is pretty close to something we already do. It is a relatively common practice in integrated pest management to leave refugia for pest species. Unsprayed crops are maintained so that the insecticide sensitive pest individuals can breed and hopefully slow down the evolution of resistance. Indeed the FDA requires that no more than 80% of a corn field be planted with Bt-Corn (corn with the Bacillus thuringiensis gene)http://www.bt.ucsd.edu/crop_refuge.html) http://www.bt.ucsd.edu/crop_refuge.html)
I really suspect that the scenario of mating structure being larger than selection structure may be that simple. In effect the larger mating structure simply lowers the intensity of selection and slows down the response to selection. Of course with lower intensity of selection, particularly with insecticides, comes qualitatively different responses to selection. The intense selection seen in the early stages of insecticide spraying appears to select for single gene resistance, where as lower rates of mortality appear to select for a more quantitative response to selection.
More interesting is what happens when the selection structure is larger than the mating structure. Here we are imagining localized mating, and selection acting over a much larger area. My first thought was that Hamiltonian sex ratios would be an example of this, but of course that is not true. Female biased sex ratios are a function of multilevel selection acting at the level of the mating unit. Rather I am thinking about a situation in which there is gene interaction. In this case the localized mating could result in alleles at interacting loci becoming associated. At a single locus this would result in the Wahlund effect, that is an excess of homozygotes, and a dearth of heterozygotes. At multiple loci it could result in the development of what might be called gene associations: Sets of interacting alleles at different loci that by random chance become associated with each other. If we imagine a field with a plant with localized mating structure, it could potentially become a mosaic of these gene associations (as well as being dominated by homozygotes within loci). Because mating is localized these associations would become heritable, and the effective additive genetic variance would be greater than the Fisherian additive genetic variance. Now our herbivore, say a cow, is wandering over this field and selecting those plants that are most palatable. It is likely that some of these gene associations would be better tasting, and the focus our cows attention (to their detriment), and others would be less palatable, and the cow would avoid them. These less palatable patches could then spread bringing their gene association with theme (dare we now call it an adaptive gene complex?).
Note that in this scenario selection is strictly at the individual level. The cow chooses the best grass to eat, but because mating is localized selection is able to build adaptive gene complexes in a way that would not be possible if the selection structure and the mating structure were matched.