The ultimate goal of system identification is the identification of possibly nonlinear systems in the presence of unknown deterministic and stochastic noise using robust, efficient algorithms that are amenable to recursive implementation. Throughout the dissertation, we will consider incrementally more difficult problems in system identification, with the aim of achieving this goal. Specifically, we begin by considering the simplest case of identifying linear systems with no noise. Afterwards, we allow for deterministic noise, followed by stochastic noise. Finally, we conclude by allowing for an unknown Hammerstein nonlinearity, before attempting to solve the problem of identifying a more general class of nonlinear systems.