Don't load all the matrices at one time.
Simply have a matrix directory, which lists all the matrice by filename and ID.
Load only the matrices the user can see, and delete the ones they can't, as they move.
Let's say the user in the middle of a matrix, for example, and the matrices are relatively small (would have to be for this to work). When the user goes forward, say, ten tiles, so that the matrix behind them can't be seen, it is deleted. Then, they run to the boarder of that matrix. The next one will be visible, they run halfway across. The next one after that is loaded. Ten more tiles forward, the one behind them is deleted, and so on.
Only thing is, like I said, matrices have to be small for this to work effectively. Reason being, you would have 5 matrices loaded at a time: The one the user is on, and the ones surronding that one.
Next step is only to set your cameras so that they can't see beyond the loaded matrices. Fog and objects on the may come in handy for that purpose, as well.