We study ideals of Novikov algebras and Novikov structures on finite-dimensional Lie algebras. We present the first example of a three-step nilpotent Lie algebra which does not admit a Novikov structure. On the other hand we show that any free three-step nilpotent Lie algebra admits a Novikov structure. We study the existence question also for Lie algebras of triangular matrices. Finally we show that there are families of Lie algebras of arbitrary high solvability class which admit Novikov structures.