My rule of thumb when Excel is still just fine:

1. The data fits on the screen. Roughly 40x30 matrices are fine. These rules aren’t set in stone, so there are situations when it’s still justified to use matrices larger than that.
2. You can see all the sheets you need. Roughly 10-14 sheets. If you need more sheets than that, you should probably split the calculation into several files.
3. You need complicated VBA macros and you need them to run perfectly every time. In my experience, this programming language is infuriatingly unpredictable and unreliable. Random things happen all the time and no amount of debugging is able to solve these types of cosmic problems.

You can go beyond those general guidelines, but using everything gets more and more annoying the further you go. Pretty soon you’ll realize it would have been better to build the whole thing in Python, R or something else. Once the file size hits 15 MB you know you’ve gone way too far and it’s about time you rebuilt the whole calculation using some nicer tool. I try to switch as soon as possible when I realize my calculations are about to grow beyond these limits.