The problem with your gantry is the thickness of it.

40mm is not a lot, so deflection is definetly possible in this.
The z plate looks to me, to sit way down from the bearings on the Z acis rails. This (even though it's 20mm thick) can also cause deflection.

What is your tabletop made from?? I see no tabletop on your router in that cideo.