In our earlier work on the evolution of representation size we stressed the importance of individuals with the same fitness as their parents', showing increase in average size in the later stages of evolution could in many cases be ascribed to them dominating the population. In Sect. 4 we introduce a fitness based penalty on programs which don't innovate. Even very large penalties produce only slight reductions in the best of run performance and, in these experiments, cut bloat by about a half.
In the experiments in Sect. 5 we have broaden research into bloat to consider non-static fitness functions. In these experiments a dynamic fitness function also cuts bloat by about a half. We also report combining our dynamic fitness function with the plagiarism penalty and note this can also produce only a small reduction in the best of run performance but can change the nature of the programs evolved and reduce their ability to generalise.
It is clear that suppressing the large numbers of programs produced in the later stages of conventional GP runs which all have the same performance by using a plagiarism penalty has not prevented bloat. If more detailed measurements confirm the penalty is having the desired effect, then further investigations into children which perform worse than their parents will be required.
Our second set of experiments tend to confirm some of the benefits claimed for dynamic fitness measures. E.g. every dynamic fitness run (without a plagiarism penalty) produced programs which performed better on the 50 random trails than the example program evolved on just the Santa Fe trail.
In both sets of experiments there is bloating and this is due to the positive covariance between fitness (even after the penalty has been applied) and program length in the evolving populations. We suspect (as in previous experiments with simple fitness functions) that this is due to shorter programs in the population being more effected by crossover than longer ones, i.e. their children follow the trails less well.
Further work is needed to understand how GP populations are able to maintain their peak performance even when the selection function appears to prevent direct copying from the best of one generation to the next. This would appear to require constant innovation on the part of the population. Current GP can ``run out of steam'' so that GP populations stop producing improved solutions [Lan96, pages 216--219,]. Therefore techniques which encourage constant innovation are potentially very interesting.