- This topic has 47 replies, 6 voices, and was last updated 2 years, 11 months ago by
.
-
Posts
-
Maxim – Thanks, I’m almost there. I had forgotten the “Get comparison stars” step! Now I get a fit. The only problem remaining is when I click on the object the flux/magnitude etc are all zero. What am I doing wrong?
One other question – Am I right that the ensemble fit graph is upside down, i.e. the brightest magnitudes are in the lower left corner. Not a problem but I would just like to understand.
Nick, do you mind sending me the FITS file with this data? You are likely clicking on sources that didn’t have any photometry fits at all, i.e. appearing without circles surrounding them. This means that the IRAFStarFinder procedure didn’t detect a point source at this location. You should get a full photometry result if you click on a source with circle surrounding it. I initially thought that it could be that some of your sources are below the default 10-sigma detection limit, but I can see that some of the brighter sources weren’t detected as well, so we probably need to introduce another adjustment to the algorithm. Maybe something up with roundness. Would like to play with this FITS data to resolve this.
On the ensemble fit – yes, you are correct, the brightest magnitudes are in the lower left corner. Do you think we should invert the axes?
Hi Nick,
Fortunately no additional setting are required – I wanted to keep the program with as little settings as possible to avoid overcomplicating it. Looks like the PSFs are sharper in your image than from the telescopes I typically use, so try setting Lower Bound for Sharpness to 0.2 and redoing photometry. It detects most of the sources in your image this way. If you want to go a little deeper, lower the Star Detection Threshold from 10 to, say, 5 sigmas. This detects pretty much all the sources down to SNR < 1, except for a faint extended source under and to the left of the galaxy.
I will next add a feature to save settings so you wouldn’t have to manually adjust things each time. I think it would be beneficial for user to save their settings once and reuse them.
Please let me know if it works.
Maxim
Thanks. That works very well. I get a pretty good ensemble fit against Gaia G and a magnitude for SN 2021hpr of 14.21. This compares to 14.29 using my aperture photometry tool, presumably affected by the galaxy background. I think the SNR you quote is too low though. I get 168, you have 27. SNR is a difficult thing to calculate correctly. How do you do it?
There is an old discussion on calculating SNR here. Basically I ended up calculating all of the non-photon sources directly by measuring the RMS value in the sky estimation. To calculate the photon, sqrt(N), noise you need to know the camera gain. I’ve implemented this in an automated aperture photometry tool that I use and it seems reasonable to me.
Nick, many thanks for the information. With PSF photometry the sky annuli are not used at all, so implementing this approach directly is not possible, unless we introduce a sky annulus setting. But then, if we do this, wouldn’t such uncertainty estimate be fundamentally detached from the photometry algorithm used?
Fortunately, I have found that the iterative PSF photometry algorithm implementation in photutils provides flux uncertainties as output. Here is the link to version 0.14 of the program that reports total error = sqrt(flux_unc^2 + fit_unc^2) instead of relying on an external SNR estimate:
http://trafyx.com/wp-content/uploads/2021/04/metropsf-0.14.zip
I don’t think I have the necessary expertise to develop a new method to estimate uncertainties with PSF fitting photometry. It should be safe to use the flux fit uncertainty provided by the underlying algorithm. I wonder if you agree with this approach and if you could give this version a try.
Maxim
Maxim,
Just to let you know that I’ve been using the program but haven’t had a chance to look at the SNR details yet. I’ll do that when I get time.
Nick.
Hi there,
Not quite sure I understand what your code does.
Differential photometry using Poissonian/Gaussian profile and then using gaia DR2 to get the magnitude of the reference star?
How does the fitted result differ from what you would get using DAOPHOT?
Also, do you set a value that allows stars that are nearing saturation to be ignored?
Is the star to be measured denoted by hand or by RA/Dec somehow?
It does look like code people would find useful.
Hi Grant,
Yes, fitting à la DAOPHOT, then deriving differential magnitudes via linear regression fit to an ensemble obtained from a VizieR catalog. You can select the catalog you need. The result shouldn’t be very much different from DAOPHOT, albeit I’m using its implementation in Astropy’s photutils. So, MetroPSF is a convenient GUI to photutils with some add-ons like linear regression fitting, plate solving, source matching, etc.
You click on the source you’d like to measure in the window. Here is a short write-up on functionality, check the Quick Start section here:
http://trafyx.com/wp-content/uploads/2021/04/metropsf-3.pdfNear-saturation are not ignored, but you can choose Gaussian sigma weighting for the ensemble, so PSF fits with high sigmas which generally appear with oversaturated sources, will be assigned less weight in the fit. Would you recommend to introduce “hard cut-off” for oversaturated stars?
Max
P.S. Grant, thinking this over again, an obvious manual solution to avoid oversaturation is to limit source magnitudes in the ensemble, which is possible in MetroPSF already. This can be combined with sigma weighting as well.
I think the version I use is a standard conversion of the DAOFIND routine to Python. I use it to provide the positions of the stars and then do photometry on them.
After that I compare my coordinates with those of stars in Gaia DR2 and generate the matches. So I then have measured flux versus catalogue mag.
I then fit a linear regression and recursively remove the outliers. I’m fairly sure I found that more successful than using weightings. I had expected extreme colour index stars to cause problems too but that had a relatively minor impact.
I think in my process any star with a peak brightness >50,000 was excluded from the linear regression. As you say setting a magnitude limit should work but the count was easy.
I look at which point is furthest from the generated fit and remove that.
Then I recalculate the fit and repeat until a decent regression coefficient is achieved and/or errors are below a threshold and sufficient stars remain.
Its not an ideal approach and has trouble when there are few stars in the scene, but the results I got suggested it was doing quite a good job – certainly much better than not culling the outliers. It may not make a huge difference generally, but its easy to code up and test so possibly worth a look as a potential refinement.
Had expected stars with the most extreme colour indices would be causing the data points far from the curve, but while they were not great they were not always the worst outliers.
Could probably dig out the code if you wanted it. I was using Gaia DR2 and cooled sensor data captured using a normal Silicon CCD.
Grant, many thanks for this suggestion. Simple and effective. I have just implemented this, will appear in version 0.15.
Happy to announce the latest version of MetroPSF, 0.15. Lots of new stuff:
• Aperture photometry.
This is similar to common aperture photometry algorithms found in other software except that a 2D model of the background is subtracted from the image, so sky annuli are not used.• AAVSO report generation.
MetroPSF can now generate AAVSO-compatible report files.• Automatic reporting on all VSX sources in the image.
MetroPSF can generate reports on all VSX sources found in the image. It cycles through each VSX variable and builds a report file. Although I would not recommend to do automatic reports and submit them blindly (unless you know what exactly you are doing) this is helpful when you need to process multiple FITS files to create a report on a single object or multiple objects with, say, similar magnitudes.• Batch processing of all FITS files in a folder.
Now it is possible to do photometric series, as MetroPSF can process multiple FITS files in a folder at once…• Light curve generation from BAA reports.
…and display a light curve from all the report files generated.• Added ability to remove outliers from the fit – by ensemble limit and maximum separation.
Much more precise photometry, as you can set it up to automatically exclude comparison stars farther than R arcseconds from the target, or that are not fitting well into the comparison ensemble, e.g. limit ensemble to N stars.Convenience changes:
• Reporting number of ensemble stars on the linear regression fit plot.
• Object name in the left panel is now automatically set from VSX catalog for matching sources.
• Display next VSX source.
Can easily cycle through all known variables in your image.• Mouse click now reports time in ISO UTC format along with photometry result.
To copy-paste in forums and discussions.Other changes:
• Increased Astrometry.Net default solve timeout from 120 to 360 s.
• Linear regression fit error is now based on standard deviation.
• MetroPSF now writes a log to metropsf.log.
• Astrometry.net URL and API key setting in the interface.
• PSF fitting algorithm setting.
• FITS crop option.Latest manual: http://trafyx.com/wp-content/uploads/2021/05/metropsf.pdf
MetroPSF program and manual: http://trafyx.com/wp-content/uploads/2021/05/metropsf-0.15.zipMax
Hi Max,
I’ve downloaded 0.15 and am running it under Windows 10. It does the PSF photometry but then fails when I try to download catalogue stars:
694 ‘charmap’ codec can’t encode character ‘u03b1’ in position 25: character maps to <undefined>
Any idea what this might be?
Nick.
Whoops. Forgot to specify encoding for the log file which obviously produces an error on Windows. Please try this fixed version:
http://trafyx.com/wp-content/uploads/2021/05/metropsf-0.15.zip
Max, Thanks. That works and this looks really powerful, particularly the ability to measure multiple variables in a field and generate automatic reports. My only remaining comment is that I get strange timestamps:
ES UMa, -4713-11-24 12:00:00.000 UTC, G = 10.91 ± 0.11
-4713 is something like JD = 0. The FITS file I’m measuring includes the record:
DATE-OBS= ‘2021-05-25T22:53:18’ / Start time of stacked exposure
Nick.
- You must be logged in to reply to this topic.
Nick, I am using SNR = Peak Flux at the centroid / Median Sky Background Level, where Peak Flux is taken from the raw FITS data, and not estimated from the photometric fit, and the sky background is taken over the entire image. Do we need any adjustments here? I wonder how 168 was derived.
Note I was able to improve the uncertainty by limiting the ensemble range to 13-16 mag – see attachment.