I have become concerned that the selection process for JBAA papers may not be "fit for purpose". My concerns are in two areas:-

1) It is clear that only a minority of the items currently published as JBAA papers actually fall into the accepted definition of a scientific or learned paper - "a written and published report describing original research results" - and therefore should not be described as such. This is not to say that these submissions should not be published, just not as JBAA papers.

2) Based on recent personal experience, it would seem that the process by which JBAA paper submissions are scrutinised (peer review or refereeing) may not be operating in accordance with accepted norms. For example, the results of the review of my paper were given only as a brief summary of a few points made rather than the entire referee's report being returned to me, as would have been expected. In addition, there was no indication that the referees had made any attempt to suggest how the paper might have been improved for publication (as they are required to do). On the other hand, they did comment on issues which, were they truly independent of the JBAA, were outside their sphere of competence: whether the paper was suitable for the JBAA and whether it would be of interest to the readeship [which is, in any case, irrelevant in the case of a scientific paper]. Finally, there is circumstantial evidence that one of the referees may have been a person with whom I had had extensive discussions about the work being reported: he should thus have declined to review the paper on the ground of prior involvement.

I attempted via several emails to raise the second area of concern firstly with the Papers Secretary and then with the Editor of the JBAA but on all occasions received a steadfast refusal to even concede the possibility that dialogue might be entered into, let alone to actually discuss the substantive issues (despite having made it perfectly clear I wished to discuss the general issues, not my paper specifically). Given this response, I felt it would be a waste of time to even raise the first concern.

I thus feel I have no alternative but to flag up the matter via the Forum, to see what the Members might have to say on these topics. Personal experience of papers submission (both good and bad, via the BAA and otherwise) would be of interest, together with any views on which types of submission should be deemed acceptable.

Over to you!

Steve Holmes

I have two papers waiting to be published, and in both cases the referees offered constructive advice in all points raised. I received complete detailed reports from both referees, who were experts in the field relative to both papers. The papers secretary was helpful with his advice and quick to respond to e-mails. I have no complaints in this respect.

My only gripe is the time it takes to get papers actually published in the Journal (about one year I think), but this means that the BAAJ has a healthy supply of material waiting to be published. A better scenario than not enough!

I have no problem at all with the type of paper which gets through for publication. The BAAJ is readable and varied. Many of the papers are section reports and results from original research (just look at the volume of Variable Star reports over the past few years). In my opinion the quality is extremely high. A top quality Journal.

Gary

I concur with Gary’s comments. I have submitted perhaps more than my fair share of papers to JBAA and have always felt the way my papers have been treated has been professional and constructive (and similar to my experience in publishing papers in peer-reviewed journals in a completely different branch of science). What has impressed me most is the willingness of referees to provide constructive advice (criticism?) that has helped me to understand and learn more.

The referees are individuals, so the type and level of feedback does vary. Sometimes there are detailed reports, other times they are more brief. And the feedback can come from different angles. That’s the nature of the best.

The Papers Secretary and the Editor have always been most helpful in dealing with my papers.

Regarding the balance of the Journal – I like the mix refereed papers,observing updates, less formal articles and news items. The Journal is the house magazine of our Association as well as providing a venue for publishing the results of original research.

Go well!
Jeremy

My thanks to Gary and Jeremy for their comments. It's good to know that accepted norms are being applied in some instances! However, while I am gratified that they have had good experiences with the papers submission process, one might perhaps say that the application of these norms seems somewhat patchy. As previously remarked, my experience was the diametric opposite from theirs: failure to return the full reports to me; no indication that the referees made constructive comments; a slow turn-round for emails, and a totally unhelpful attitude from the Papers Secretary. All of which confirms my view that I have been badly treated and that the questions I posed to the Papers Secretary were entirely justified and worthy of a reply in a professional manner.

I would agree with Jeremy that different referees may give different levels of comment if left to their own devices, but there are very clear guidelines for referees out there on the Internet and so there really is no excuse for this sort of inconsistency. Inconsistency in the treatment of authors is equally inexcusable.

Moving on to the concept of a "JBAA paper", I am also very appreciative of the mix of articles published and would agree that the Journal is a top-quality publication. This was not my point, however. I was observing that a large percentage of the articles published as JBAA papers should not have been so classified as they do not fall within the definition of a scientific or learned paper. Publish them elsewhere in the Journal by all means but defining what are, in some cases, merely general interest articles as JBAA papers will only serve to debase the whole concept of the Journal Paper as an entity of scientific merit. One should never confuse the roles of referee and proof-reader!

On Gary's specific mention of section reports, while they should clearly form part of the Journal (as they record the extensive work done by many BAA members over a period of time), I would submit that they also are not "scientific papers" in the true sense of the word. The main problem with them is that they are almost impossible to referee: the source data will rarely be available and there is no logical flow of thought process from theory to observation to conclusion. They are simply what they say they are - summary reports of a season's observations, no more and no less. Their credibility is not improved by an attempt to dress them up as anything else. There is also a great deal of inconsistency in this area, incidentally - for example, can anyone show me the difference between Richard McKim's paper on Mars in 2005 in Vol.121 No.4 and his essentially identical-in-concept reports on Venus in Vol.121 No.6, Vol.122 No.1 and Vol.122 No.2 ?

While researching the background to my comments I performed a review of all items published as JBAA papers over the last few years and worked this up into a document (I almost said "paper"!) in which I analyse their status as "papers" or "non-papers". If anyone is interested I can finish this off and submit it to the Forum so members can have a fuller appreciation of my concerns in this area.

Steve Holmes

One of my two papers 'in waiting' is a VSS section report concerning the Polar Programme I co-ordinate. The paper was refereed by one professional and one amateur astronomer. I would like to think that it is a 'scientific' paper as such, as it reports on BAAVSS data on objects which are of interest to the professional community. It also presents data in the form of light curves which is probably not available anywhere else. I think to say that a section report can't be scientific (depending on it's original purpose and how it's presented of course) is a bit unfair on both the Director of the section concerned and the observers who have contributed.

Just a final thought from me on this. I wonder if this is the correct place to voice your thoughts on the paper secretary and journal editor, who both do an extremely difficult job exceedingly well. It's unlikely they will repsond on this forum, so it's a bit of a one sided debate. Surely a letter to Council would be more appropriate.

Just my view!

Gary

I think your parenthesised comment is relevant here - it depends entirely on what the report is reporting. Maybe I should have been a little more precise with my terminology earlier - when referring to a "section report" I was really meaning a report of the summary variety (such as those by Richard McKim I mentioned) rather than of the "new data" variety that you seem to be alluding to. A report on new collections of data leading to the production of new or updated light-curves could well fall into the scientific paper category (much as those by Jeremy and his associates clearly do).

The real essence of whether a submission is an article or a paper is, as mentioned in my last post, whether it can be validly peer-reviewed. Peer review is the process by which the technical approach; methods; observations; data collection & reduction techniques, and conclusions of the author(s) can be assessed [and, where relevant, to check that the work reported is not falsified or plagiarised]. It therefore follows that if the submission being refereed does not have all or most of these elements it will be difficult to peer-review it. This is why I find it difficult to concede that "summary-type" reports can be regarded as scientific papers in the fullest sense of the word, and why the many "general interest" articles currently classified as papers are certainly not such.

As to whether the Forum is the correct place to be discussing these matters, I did of course think long and hard about this. In the end I decided I should "go public" as I felt it was important to solicit the views of the general membership, as they were likely to have a rather more independent view of things than Council members. Also, I could only write from my own experience so I felt it would be helpful to be able to put this into context. Thirdly, based on the uncompromising replies received during my initial attempts at discussion, I was unconvinced that approaching Council would be worthwhile - this might be called pre-judging the case but by this time I was getting rather frustrated by the lack of dialogue! Lastly, it was suggested to me during these interchanges that I should start a Forum topic on the matter, so eventually I did. Maybe not the correct judgement, but those were my reasons.

Steve Holmes