Paul Leyland

Two standard approaches, one lazy and the other more rigorous.

Lazy: guess the width of the signal peaks and set the smoothing window to be around 3 times that. Use degree 2 or 3 polynomials unless the window is very large (over 20 say) when you could choose degree 4 or 5.

Rigorous: Compute residuals for a range of window widths and polynomial degrees. Perform statistical tests on them and choose the filter which best passes your acceptance requirements of high noise to signal ratio (that way round because you are trying to remove all the signal from the noise).  The Durbin-Watson statistic, which measures autocorrelatIon at lag 1, is a simple test which tends to work well, but a fuller autocorrelation at a variety of lags might be more appropriate.

Wikipedia has good articles on S-G, D-W and several other tests for signal in the presence of noise. Easy to use implementations are all over the place. I tend to use R as it is free and portable, unlike Matlab and Mathematica.