FITS and FIT files suffix

Forums Photometry FITS and FIT files suffix

Viewing 17 posts - 1 through 17 (of 17 total)
  • Author
    Posts
  • #621359
    Kevin West
    Participant

    Is there any difference between these? ASTAP calibration outputs a FITS file which will not upload to AAVSO VPhot reduction software, whereas the raw files created by AISair controller upload fine.
    Kevin

    #621361
    Dr Paul Leyland
    Participant

    The FITS standard defines a format for metadata headers and mandates some of them.

    Within reason, you can put pretty much anything in there. Whether a particular FITS-reading program interprets them in an intended manner is an entirely different matter.

    For instance, whether either, neither, or both of “EXPOSURE” or “EXPTIME” is present depends on the writing program. My code to read FITS files has to be able to handle both and to work out what to do if their values differ.

    #621376
    Kevin West
    Participant

    Thanks Paul,
    Does the actual file name suffix make any difference.
    The raw files are .FIT and upload fine
    The ASTAB calibrated files end in .FITS and will not load.
    It’s a real shame because I have spent many hours and struggled to get any where calibrating with AIP4WIN, ASTROImJ and DeepSkyStacker.
    ASTAP is very user friendly and seemed to work very smoothly until I hit this problem.
    Kevin

    #621377
    Nick James
    Participant

    The file extension shouldn’t make any difference.

    Are the files roughly the same size? The raw files you get from the camera are probably 16-bit integer (BITPIX=16). After calibration they may be floating point (BITPIX=-32) and it may be that Vphot can’t cope with that. The FP files will be twice the size of the integer ones. If Vphot can’t cope with FP that is a bit poor but there may be an option in ASTAP to save as ints.

    Failing that, have a look at the FITS header and see if you can spot any significant differences. That might give a clue as to what is going wrong.

    #621378
    Dr Paul Leyland
    Participant

    Are the files roughly the same size? The raw files you get from the camera are probably 16-bit integer (BITPIX=16). After calibration they may be floating point (BITPIX=-32) and it may be that Vphot can’t cope with that. The FP files will be twice the size of the integer ones. If Vphot can’t cope with FP that is a bit poor but there may be an option in ASTAP to save as ints.

    I hit exactly that problem with Astrometrica. To solve it I wrote a Perl script to convert from floating point back to 16-bit unsigned integers. Not entirely trivial because a FP number can be negative and the range can be greater than supported in 16 bits, so off-setting and scaling was required.

    #621393
    Grant Privett
    Participant

    I’ve seen FITS format files that were FIT, FITS, FTS and f and lower case variants thereof. Never encountered any that were not processed.

    Have once come across software that seemed only equipped for 16 bit data – a histogram of 65536 bins was being used.

    Theres some fun for anyone using Python, the Astropy library discourages the writing of 16 bit FITS files and I think it defaults to 32bit.

    #621397
    Kevin West
    Participant

    Thanks for the tips and leads on this problem. The files are indeed converted to (BITPIX=-32) by ASTAP and doubled
    in size being 65Meg.
    AAVSO people have advised me to check that FITS header details for my system correspond to registration info on VPhot
    which it didn’t. I corrected this expecting success but sadly no.
    I just want to upload my obs. Very frustrating.
    If anyone can spare the time I’ll upload the FITS header.
    The uncalibrated files from ASIair load fine
    The stacked and calibrated file from ASTAP will not
    Kevin

    #621400
    Nick James
    Participant

    It looks as if it is something to do with the data format. A lot of programs can read FP files now (even Astrometrica) but they assume that the data is scaled in the same way as an integer format file, i.e. a range of 0..65535 and they just truncate to the nearest integer. FP format FITS files can have some wacky scalings. I’ve seen some where the entire image dynamic range is in the range of 0.0 to 1.0 and that clearly doesn’t work very well if you just truncate. From looking at the header of your calibrated file that does not look to be the case here.

    The headers look OK with nothing obvious. Do you get any hint in the error message from VPHOT or does it just silently die? As a workaround can you get ASTAP to save your calibrated files in integer format? I would have thought that there would be some option to do this.

    #621401
    Grant Privett
    Participant

    Silly thought, is it because the FITS header is rather longer. I think it has to be multiples of 2880 bytes or something. I don’t suppose the author of Vphot could have made the assumption that it was always 2880? Saw that once. But only once and 10 years ago.

    #621402
    Nick James
    Participant

    I would hope they use a standard library (e.g. cfitsio) to read and write FITS files rather than something homebrew. I use cfitsio for all of my code and it is very robust.

    #621412
    AlanM
    Participant

    ASTAP does allow saving as a 16 bit FITS file:

    File -> Save as FITS file

    On the Save file as window that appears select the down arrow on the Save as type field. One of the options is 16 bit FITS files (*.fit*).
    Select this and change the name of the file as required and save.

    Theres some fun for anyone using Python, the Astropy library discourages the writing of 16 bit FITS files and I think it defaults to 32bit.

    I have some memory of being able to convert 32 bit files to 16 bit in Python by using numpy (get the data into a numpy array and convert using np.int16(data) and then write the data to a file) but I could be mistaken.

    #621443
    Grant Privett
    Participant

    Will let you know in a few days. Hope to have a play this weekend.

    I think I last had a play a year or so back and I did the numpy int16 thing (taking care that the values fit in 65536 bins) and it just wrote out a BITPIX=32. I have a feeling I have to force its hand with BSCALE and BZERO explicitly plus there is some option for in effect, “Don’t mess about with BZERO and BSCALE”. It may be you have to – yourself – make sure the image counts are in the range -32768 to 32767…

    If you had code to hand I would be keen to see it.

    #621444
    Dr Paul Leyland
    Participant

    Ah, the beauty of standards!

    It is very important, IMAO, to have a standard. So important that everyone should have one of their own.

    #621445
    Dr Paul Leyland
    Participant

    I’ve seen some where the entire image dynamic range is in the range of 0.0 to 1.0 and that clearly doesn’t work very well if you just truncate.

    That is relatively sane compared with some formats I have seen. SWarp, in particular, can generate values which are negative and in a range which bears little resemblance to that of 16-bit signed integers. The result is fully compliant with the FITS standard and, for example, ds9 has no trouble dealing with such images.

    That’s why I had to write my own conversion routine.

    #621448
    Nick James
    Participant

    According to this thread from six years ago:

    https://www.aavso.org/reducing-fits-size-use-vphot

    Vphot has an upload file size limit of 50 MB. That is probably why your attempt to upload the FP images fails since the filesize is 65 MB. I would have thought that it would tell you this rather than failing silently.

    #621449
    AlanM
    Participant

    It was something I thought I’d seen on the internet but your concerns make me think it was related to something else. Maybe converting a jpeg to 16 bit fit. I’ll have a look to see if I can find the page but it was sometime ago when I saw it.

    #621555
    Kevin West
    Participant

    Thanks for the inputs on this. I finally found out that it’s the non-letter characters in the file name created by ASTAP
    E.G it sprays in commas and brackets etc.
    Took these out and it worked.
    I susected the files size too at first but it’s 200Mb limit.
    I don’t know why it’s necessary for ASTAP to resize the files. The originals prior to stacking and calib were all 32Mb
    Thanks
    Kevin

Viewing 17 posts - 1 through 17 (of 17 total)
  • You must be logged in to reply to this topic.