Blending 3D models with videos

Currently, we leave the work of capturing detailed videos to professional videographers & building complex & intricate 3D animated scenes to the likes of Pixar & company who take a painstaking effort, time & money in creating masterpieces. In fact even games are getting so realistic that they would be beating Pixar & company pretty soon. However, what we need is a simple conversion utility which will convert a video into a 3D world in which 3D characters can move about realizing the presence of walls and should be able to interact with the objects present in the videos. According to me this blend between videos & 3D animation is an interesting angle to research as it would open an exciting possibilities. Remeber playing Sims type game an walking down Champs Elysees with your date and making out with her, with the Eiffel Tower in the background. And obviously the woo-hoo can be done somewhere or the other :-P