Few games in the Castlevania series have been cancelled -- at least, not the ones fans know about. Certainly there are probably a few game ideas that have been pitched and then thrown out before they ever reach true developement stages, but of the games that have been announced in some form, only a handful have been left unreleased.
In 2008, Konami announced a new game in the franchise, a proposed reboot of the series starring Alucard. The game, called simply Akumajo Dracula (the Japanese name for the series, meaning Demon Castle Dracula) the game would have restarted the series fresh, throwing out the history of what came before (well, supposedly -- this is largely conjecture at this point based on one released trailer and comments from Konami).
And then after its announcement, nothing. The game disappeared from radar and was then never heard of again.
Legacy of Castlevania (2008):
So what happened to Castlevania 2008 (the title we'll give this for the presumed American release it would have gotten)? Probably whatever engine was developed for this game was worked into the eventual reboot the series saw, Castlevania: Lords of Shadow. Games take a while to develop, and while the game was announced in 2008, the direction could have shifted to focus on Belmonts and Alucard was swapped out for a while-wielding hero. That game was released for the same systems Castlevania 2008 was announced for -- Lords of Shadow just took its spot on the release schedule.
Not that all the ideas from this game were probably dropped. Alucard would eventually show up in the reboot continuity so it's entirely possible the some ideas and nots from this first attempt at a reboot were saved for the 3DS entry, Lords of Shadow - Mirror of Fate. Plainly Konami ddn't want to completely get rid of all continuity, so characters and concepts from the previous games were still worked into the eventual reboots that were developed (because, really, would it be a Castlevania game without a Belmont, or Alucard, or at least Dracula?).
Simply put, though, no one really knows why the game vanished. Whatever happened to the game, however it evolved or was just simply dropped, it's unlikely this version of Castlevania will ever see any further developments beyond it's one teaser trailer.