We consider a type of overlearning typical of independent component analysis algorithms. These can be seen to minimize the mutual information between source estimates. The overlearning causes spikelike signals if there are too few samples or there is a considerable amount of noise present. It is argued that if the data has flicker noise the problem is more severe and is better characterized by bumps instead of spikes. The problem is demonstrated using recorded magnetoencephalographic signals. Several methods are suggested that attempt to solve the overlearning problem or, at least, diminish reduce its effects.