You could expand on your box collision - like if your floor is a big box, then the ladder can be surrounded by another box. You'd check for collision with the ladder box and make it so you can move up and down inside it. For good control it's nice if your ladder is like a sorta portal where you can move in 3D, but step too far away from the ladder and it's outside the box again and he falls. Normally I'd run straight for a ladder then look up and expect to zip up the ladder - if you can get it to do that then your laughing.
Perhaps you'll need a maths collision method though. If you store the location and size of your collision box in an array - you could make it so a function simply returns the collision flag, you'd call a function with the player coordinates, and it returns the flag. You could give each box a type too, so type 1 might be a solid box, type 2 might be thin air, type 3 might be a ladder, type 4 might be a light switch etc etc. The cool thing is that using this style of collision means you can make a box then cut a hole in it, making doors is quick and easy - makes for easy lift raising and lowering too - because you only need to assign the box data to the actual 3D object position and the system would do the rest. Also, besides just returning a collision flag, I strongly suggest setting global variables to the return location of the collision, so the player height is taken from the collision data - makes things very smooth and helps avoid those falling through floor bugs.
Van-B
My cats breath smells of cat food.