By James C. Rautio
Part II of this column can be found here.
Part III of this column can be found here.
“It’s just an Old Boy’s Club,” my host firmly told me. We were talking about getting papers accepted to the IMS (International Microwave Symposium). After several rejections, he no longer submitted papers to IMS. This is a real shame because IMS is the premier RF/microwave conference in the world. His perception was that the IMS TPRC (Technical Paper Review Committee), consisting of more than 250 of the world’s leading RF/microwave researchers, gave favored status to its own members.
I have been on the IMS TPRC for a good share of the last quarter century, sitting on several different subcommittees during that time. I can say absolutely and without hesitation (with one single exception, mentioned below) that there is absolutely no overt, not even wink-of-the-eye bias toward or away from committee members for paper selection. There is also no such bias for or against any nationality, race, religion, etc. None. Nada. Period. Just to underline that fact, I, and all of my friends on the committee, have had multiple papers rejected by the IMS. You know, it happens. We just have to deal with it.
How about that one exception? During IMS2011 TPRC, a member of another subcommittee approached my subcommittee and lobbied for us to reconsider a rejected paper, presumably written by a buddy. We politely told him that reconsidering would be an unfair advantage for his “buddy” and we would not do it. This was the only (attempted) exception in more than 25 years. Not bad. The IMS TPRC is a really good crew of very smart folks.
However, could there be possible subconscious bias? I mean, we are all human. The example I like to use is the introduction of New Coke a few years ago. In double-blind taste testing, Old Coke was consistently losing to Pepsi. They formulated New Coke so that it consistently won double-blind taste testing. But the New Coke introduction was a disaster. People everywhere were dumping New Coke for Pepsi. Why?
After careful research they determined one possible cause was the unique taste combined with the red can . Take away the red can in double blind, and Old Coke loses. With the red can, Old Coke wins. It was entirely subconscious.
Do papers wrapped in a TPRC member’s byline have a subconscious advantage? We don’t know, and we really can’t run a statistically significant experiment to test that hypothesis. And don’t forget, we are scientists and engineers. We do not under any circumstances draw firm conclusions from anecdotal incidents. Whether or not the subconscious bias is real, IMS2011 decided to have all papers submitted in a double-blind fashion. Properly written, there is no clue in the paper anywhere as to who the authors are. When the reviewers cannot otherwise figure out who wrote the paper, there is no chance of real, imagined, or subconscious bias in paper selection. In these cases, papers can be selected based only on perceived merit.
Keep in mind that every year it is up to each year’s TPC chair whether or not a double blind is used. So why wouldn’t they use it? Because a few authors “game” the system. For example, when under extreme pressure to publish, some might yield to the temptation to publish substantial portions of the same material multiple times. This can be OK as long as any duplicate material is clearly indicated. Even when the reviewers don’t know who the authors are, as long as the author clearly indicates exactly what has already been published and as long as there is a good amount of new material, acceptance of the paper is still possible.
However, all reviewers I know (including me) are by nature skeptical. The motto on the U.S. dollar bill is, “In God we trust.” Engineers and scientists like to add to this, “All others must have proof.” I am confident that all the papers I have ever reviewed are written by members of the “all others” category. I have seen authors do double publications. One year, I reviewed a paper for the MTT Transactions. Several months later, I found myself reviewing another paper for IMS that was almost entirely a cut-and-paste version of the first paper. Neither paper made any mention of the other. With the TPC chair’s permission, I handed both papers to the MTT Transactions editor. They deliberated and took action they felt appropriate.
For IMS2012 TPRC, two members in my group found three such papers by doing IEEE Xplore searches for keywords and phrases. We can only guess how many double publications we missed. Folks, this is not good. As I said, future IMS TPC chairs can choose whether or not they continue the double blind. Frankly, if you are a new author trying to get a first paper published at IMS, you want double blind, even if it might only be a placebo. If you know anyone who might be trying to sneak in a double publication, please encourage them to refrain from ruining it for everyone.
Notice in the above I said decisions are made based on perceived merit, not actual merit. Talk to the philosophers. We do not know what the actual world is. We only know what we perceive to be the actual world. And, at least in this universe, reviewers must make their decisions based on their perception as to what the actual merit is. Your job in getting your paper selected is to make the reviewers’ job of perceiving a high actual merit as easy as possible. In my experience, most rejected papers are rejected because the authors make that job difficult.
In my next “Reflections” column, I give a few pointers on how to make a reviewer’s job, which is to quickly see how good your paper is, as easy as possible.
Part II of this column can be found here.