The file extension shouldn’t make any difference.
Are the files roughly the same size? The raw files you get from the camera are probably 16-bit integer (BITPIX=16). After calibration they may be floating point (BITPIX=-32) and it may be that Vphot can’t cope with that. The FP files will be twice the size of the integer ones. If Vphot can’t cope with FP that is a bit poor but there may be an option in ASTAP to save as ints.
Failing that, have a look at the FITS header and see if you can spot any significant differences. That might give a clue as to what is going wrong.