It sounds like you have an idea for a FPS/RPG Game (I'm playing The Borderlands on the PS3 right now). IMO the first step would be to outline the details of the game. Once you have these details, you will be able identify features and game systems needed to build your Game Engine.
The next step will be to design the Game Engine. The Game Engine runs the game. It will integrate and manage 2D/3D Rendering, Shaders, Physics/Collisions, AI/logic, UI & Dialogs, Sound & Music, Networking. There are many technical details that should be thought out with these systems to present the players with a smooth and responsive interactive experience. So managing memory, media, user input, network communication, etc may require tweaking for high performance.
Heres a illustration of the Engine Design for the
DGDK Open Source Project Super 3D Game Engine (S3GE).
Once your Game Engine is built, you can develop Editor(s) with it to design Game Content. You will most likely require more than one Editor - more like a suite of Editors: GUI Editor, AI Editor, Particle/FX Editor, Character Editor, an editor for any form of exported/imported data.
I'm lazy and prefer to use FREE Full-powered Editor Applications. However, if your gonna build your own Editor App, it will be easier to build it using the resources from the Game Engine, providing the most accurate results at runtime. Also, smaller versions of these Editor Apps may be used in-game for Character Customization, Inventory, etc.
I understand that Terrains are top priority for you (and I'm curious as to why), but, how will the Game Engine manage and render them? Will your game require panaromic views of the landscape from any distance/angle? Are your worlds massive? If so how will the engine load/stream them? Will the terrain be deformable in realtime? Does the engine use shaders on the terrains? Many other questions. Its important to identify these requirements upfront, because how the engine uses terrain can influence how the editor generates and exports the terrain data.
Q: What came first? The Chicken or the Egg?
A: The Game Engine!
This is where I recommend you start. Build your Game Engine. DGDK simplifies creating your Game Engine, thats what makes it cool! To start building your engine, I recommend you get the following systems working together:
1. Scripting Engine - import/export data formats, macros, embedded scripting, and addons.
2. Networking - support file/data transfer and communication across internet.
3. Physics - Fast Basic Collision Calculation and complex physics simulation.
4. Pathfinding - AI controlled Collision Avoidance.
5. Particle System - High Performance Visual Effects/massively replicated 3D and 2D entities.
6. User Interface/GUI - User Applications/Game Interface.
7. Interior Rendering System - High Performance Rendering Building Interiors Geometry.
8. Exterior Rendering System - High Performance Rendering Terrain Geometry.
There are other high-level system/subsystems that can be added and your engine may not require all of the above. However, if you do, it will be a very powerful engine that you can build great Editing Apps and Games with.