- This topic has 16 replies, 5 voices, and was last updated 2 months, 3 weeks ago by
.
-
Posts
-
Is there any difference between these? ASTAP calibration outputs a FITS file which will not upload to AAVSO VPhot reduction software, whereas the raw files created by AISair controller upload fine.
KevinThe FITS standard defines a format for metadata headers and mandates some of them.
Within reason, you can put pretty much anything in there. Whether a particular FITS-reading program interprets them in an intended manner is an entirely different matter.
For instance, whether either, neither, or both of “EXPOSURE” or “EXPTIME” is present depends on the writing program. My code to read FITS files has to be able to handle both and to work out what to do if their values differ.
Thanks Paul,
Does the actual file name suffix make any difference.
The raw files are .FIT and upload fine
The ASTAB calibrated files end in .FITS and will not load.
It’s a real shame because I have spent many hours and struggled to get any where calibrating with AIP4WIN, ASTROImJ and DeepSkyStacker.
ASTAP is very user friendly and seemed to work very smoothly until I hit this problem.
KevinThe file extension shouldn’t make any difference.
Are the files roughly the same size? The raw files you get from the camera are probably 16-bit integer (BITPIX=16). After calibration they may be floating point (BITPIX=-32) and it may be that Vphot can’t cope with that. The FP files will be twice the size of the integer ones. If Vphot can’t cope with FP that is a bit poor but there may be an option in ASTAP to save as ints.
Failing that, have a look at the FITS header and see if you can spot any significant differences. That might give a clue as to what is going wrong.
Are the files roughly the same size? The raw files you get from the camera are probably 16-bit integer (BITPIX=16). After calibration they may be floating point (BITPIX=-32) and it may be that Vphot can’t cope with that. The FP files will be twice the size of the integer ones. If Vphot can’t cope with FP that is a bit poor but there may be an option in ASTAP to save as ints.
I hit exactly that problem with Astrometrica. To solve it I wrote a Perl script to convert from floating point back to 16-bit unsigned integers. Not entirely trivial because a FP number can be negative and the range can be greater than supported in 16 bits, so off-setting and scaling was required.
I’ve seen FITS format files that were FIT, FITS, FTS and f and lower case variants thereof. Never encountered any that were not processed.
Have once come across software that seemed only equipped for 16 bit data – a histogram of 65536 bins was being used.
Theres some fun for anyone using Python, the Astropy library discourages the writing of 16 bit FITS files and I think it defaults to 32bit.
Thanks for the tips and leads on this problem. The files are indeed converted to (BITPIX=-32) by ASTAP and doubled
in size being 65Meg.
AAVSO people have advised me to check that FITS header details for my system correspond to registration info on VPhot
which it didn’t. I corrected this expecting success but sadly no.
I just want to upload my obs. Very frustrating.
If anyone can spare the time I’ll upload the FITS header.
The uncalibrated files from ASIair load fine
The stacked and calibrated file from ASTAP will not
KevinIt looks as if it is something to do with the data format. A lot of programs can read FP files now (even Astrometrica) but they assume that the data is scaled in the same way as an integer format file, i.e. a range of 0..65535 and they just truncate to the nearest integer. FP format FITS files can have some wacky scalings. I’ve seen some where the entire image dynamic range is in the range of 0.0 to 1.0 and that clearly doesn’t work very well if you just truncate. From looking at the header of your calibrated file that does not look to be the case here.
The headers look OK with nothing obvious. Do you get any hint in the error message from VPHOT or does it just silently die? As a workaround can you get ASTAP to save your calibrated files in integer format? I would have thought that there would be some option to do this.
Silly thought, is it because the FITS header is rather longer. I think it has to be multiples of 2880 bytes or something. I don’t suppose the author of Vphot could have made the assumption that it was always 2880? Saw that once. But only once and 10 years ago.
I would hope they use a standard library (e.g. cfitsio) to read and write FITS files rather than something homebrew. I use cfitsio for all of my code and it is very robust.
ASTAP does allow saving as a 16 bit FITS file:
File -> Save as FITS file
On the Save file as window that appears select the down arrow on the Save as type field. One of the options is 16 bit FITS files (*.fit*).
Select this and change the name of the file as required and save.Theres some fun for anyone using Python, the Astropy library discourages the writing of 16 bit FITS files and I think it defaults to 32bit.
I have some memory of being able to convert 32 bit files to 16 bit in Python by using numpy (get the data into a numpy array and convert using np.int16(data) and then write the data to a file) but I could be mistaken.
Will let you know in a few days. Hope to have a play this weekend.
I think I last had a play a year or so back and I did the numpy int16 thing (taking care that the values fit in 65536 bins) and it just wrote out a BITPIX=32. I have a feeling I have to force its hand with BSCALE and BZERO explicitly plus there is some option for in effect, “Don’t mess about with BZERO and BSCALE”. It may be you have to – yourself – make sure the image counts are in the range -32768 to 32767…
If you had code to hand I would be keen to see it.
Ah, the beauty of standards!
It is very important, IMAO, to have a standard. So important that everyone should have one of their own.
I’ve seen some where the entire image dynamic range is in the range of 0.0 to 1.0 and that clearly doesn’t work very well if you just truncate.
That is relatively sane compared with some formats I have seen. SWarp, in particular, can generate values which are negative and in a range which bears little resemblance to that of 16-bit signed integers. The result is fully compliant with the FITS standard and, for example, ds9 has no trouble dealing with such images.
That’s why I had to write my own conversion routine.
According to this thread from six years ago:
https://www.aavso.org/reducing-fits-size-use-vphot
Vphot has an upload file size limit of 50 MB. That is probably why your attempt to upload the FP images fails since the filesize is 65 MB. I would have thought that it would tell you this rather than failing silently.
It was something I thought I’d seen on the internet but your concerns make me think it was related to something else. Maybe converting a jpeg to 16 bit fit. I’ll have a look to see if I can find the page but it was sometime ago when I saw it.
Thanks for the inputs on this. I finally found out that it’s the non-letter characters in the file name created by ASTAP
E.G it sprays in commas and brackets etc.
Took these out and it worked.
I susected the files size too at first but it’s 200Mb limit.
I don’t know why it’s necessary for ASTAP to resize the files. The originals prior to stacking and calib were all 32Mb
Thanks
Kevin
- You must be logged in to reply to this topic.