Yes, I was wrong

… you can’t make traffic go one way for one port, and another way for some other port.

Why? Because you’re most likely stuck with switched traffic that only sends traffic down one cable when it figures out a certain network-address is connected to that cable.

However, I did manage to subtract one router out of the equation with the wondrous discovery of selective NAT. Aparantly you can change the default source 0.0.0.0 ACL to a specific per port list. The ip’s and ports that don’t match the ACL won’t be translated. Which is exactly what I was looking for.

This does require that with every port forward from your isp’s router to be added to the ACL of the 2nd router, but that won’t be that much of a problem.