The simultaneous topology optimization and training of neu- ral networks is a problem widely studied in the last years, specially for feedforward models. In the case of recurrent neural networks, the existing proposals attempt to only optimize the number of hidden units, since the problem of topology optimization is more difficult due to the feedback connections in the network structure. In this work, we make a study of the effects and difficulties for the optimization of network connections, hidden neurons and network training for dynamical recurrent models. In the experimental section , the proposal is tested in time series prediction problems.