Learning to Grow and Growing to Learn...