In the case of 640×480 video where the aspect ratio is lower than that of the sensor I would expect the downsampling to be done based on the height of the sensor and then the left and right edges cropped to match the aspect ratio ie 5184×3456 is downsampled to 720×480 and then cropped by 40 pixels left and right
British Astronomical Association
Supporting amateur astronomers since 1890