I think that artists, and you are an artist if you are investing so much time and energy into creating something in representational form, often make the mistake of confusing the tool with the results. You can create a masterpiece with a sharp rock and a piece of expensive garbage with the finest tool sets known to humanity.
It's the artist stu - er - student!
Heh, heh! Joking aside it really is the artist. I settled on the Gimp for 2d work and Blender for 3d because they are both the most feature rich and complete FREEWARE applications for their niches available over the Internet and I'm the type of person that wants all the bells and whistles available even if I only ever use an eighth of them and really understand fewer than that.
None of which matter. The thing is that all the applications you mentioned will allow you to produce a perfectly acceptable looking castle like structure. Currently I am building a 1920s era single family house with Blender and having no problems at all, but FIRST I had to learn to use and to think with Blenders formidable and feature rich tool sets. The application DOES have a really steep learning curve. Had I learned to use Milkshape instead, however, I would have been just as comfortable creating 'the house that Opus built.'
Now as for crafting buildings there is a freeware product supported by The Game Makers that you can instantly download and begin using after only a day or two of study and that is Google Sketchup. I've downloaded it and spent a couple of hours studying it and I feel perfectly comfortable with the notion of using it for any and all structure work because it is meant for such work and meant to be used by people who don't know 3d applications from wart removal.
Why didn't I use it to build that 1920's style house? I am already growing comfortable with Blender and I want to go on to really get the hang of working with that application first before going on to use any other 3d apps.
BUT it's all up to the individual artist. If you have the money to buy whatever you want and are not already committed to learning any one application then I would suggest you go with either Google Sketchup or 3d World Studio because both of them have been endorsed by The Game makers and both of them are, specifically are, for building structures. It's just that Google Sketchup has a freeware version and you have to pay for 3d World Studio.
Eternal student in search of knowledge. But will settle for the occasional epiphany.