Lecture Notes 3 PDF

2010/4/8
How to get my paper accepted?

Part 3: Review, Rebuttal, Revision, & Ethics
1
C.K. TOH
VISITING CHAIR PROFESSOR
N C K U TA I N A N
08 APRIL 2010
INTERNATIONAL
FORUM
NCKU EECS College: International Forum Seminar Series for Graduate Students
INTERNATIONAL FORUM
Seminar Series Outline
2
2010/4/8
ANNOUNCEMENTS
3
y ESSAY WRITING CONTEST: Core students please
submit a 1-page essay with the title: My Thoughts on

using English in NCKU. Deadline is 12 April 2010.
Email your essay to: eecs@eembox.ee.ncku.edu.tw
y TUTORIAL HOURS: Starting from 26 April 2010,
every Monday and Wednesday. Open to EECS PhD &

Masters only.
{
{
{
{
{
{
Time: 2pm 3pm.

Place: Dr. Tohs office at ChiMei Building, 4th Floor, R95403
Advance reservation of tutorial slots needed.
Each student is allowed 20 minutes of consultation time.
Please make your reservations in advance online on web.
Your reservation will be confirmed via email sent to you.
Outline
Todays Lecture
4
y 1. Reviews
{ Evaluation Metrics Used (conferences, magazines, journals)
{ Qualitative & Quantitative Measures
{ Decisions (minor, major, accept, reject)
y 2. Rebuttal
{ How to reply to reviewers comments
{ What if reviewers are wrong?
{ What if reviewers are unreasonable?
{ When to move on?
y 3. Revision
{ First revision what you need to do..
{ Second revision possibly last
{ Third revision - unlikely
y 4. Research Ethics
y 5. Conclusion
2010/4/8
RECALL
Technical Paper
5
y Contents
{
{
{
{
{
{
{
{
{
{
{
My Paper
Abstract
Introduction / Motivation
The Problem
Your Work / Proposal
More More Your Work
Comparisons
Claims / Discoveries
HEAD/ Title
(Abstract & Introduction)
BODY
(My Work)
I proved that ..
I discovered that ..
Related Work
Future Work
Conclusion
References
TAIL
(Conclusion)
SUBMISSION
Please Check
6
Before you submit your paper, ensure it is:

y Readable
y Free of errors (language or technical)
y Within page limit (else pay $$)
y Conform to required format
{
{
{
{
Font type
Font size
Margins
Headings, sections, subsections
Readable
No Errors
Technical
substance
2010/4/8
SUBMISSION
Please Check
7
y Submission
revision
Write
Paper
No
Professor
Check
Paper
OK?
Yes
Sent out
For
Reviews
Editorin-chief
(EiC)
Submit
INTERNATIONAL FORUM
8
y1
Reviews
Review: to examine critically or to study material again
2010/4/8
REVIEWS
The Process
9
Reviewers
y Assignment of Reviewers
EiC
OR
Return scores to EiC
Read,
Check,
Evaluate,
Score
Review period can

be from 3 months up
to 1 year or more
EiC Assistant or
Associate Editor
REVIEWS
Evaluation Methods
10
y Paper Review
1
Quantitative Evaluation
introduce evaluation metrics
use numeric scores (say 1 5)
give an overall
recommendation score
2
Qualitative Evaluation
reviewers must comment on the
paper regarding various aspects,

such as:
weakness of paper
strength of paper
comments for improvement
overall recommendation
2010/4/8
REVIEWS
Evaluation Metrics
11
y Reviewers are looking for (main metrics):

{
Novelty
Significance
Did you introduce anything new in your paper?
Are your work significant or is it just marginal work?
Impact
Did your work impact the field or the community or the society?
REVIEWS
Other Evaluation Metrics
12
y Readability
{ Is your paper written in such a way that is easy to read?
y Originality
{ Does your paper contain original ideas and is not copied from elsewhere?
y Technical Correctness
{ Are there any flaws in your technical approach/solution?
y Timeliness
{ Is your topic appropriate for the current year or is it an old topic that is
fading away?
y Relevance
{ Is your topic relevant to the theme of the conference or journal?
2010/4/8
REVIEWS
Evaluation Scoring
13
y REVIEWER SCORING (see appendix for more examples)

Metrics
Score (1: lowest; 5: highest)
Novelty
Significance
Technical Correctness
Timeliness
Relevance
Overall Recommendation
Note: Some reviewers use 2.5 but this should be avoided as much as possible!
REVIEWS
Qualitative Evaluation
14
y REVIEWERS COMMENTS (see appendix for more examples)

Bad
Good
This paper is poorly organized and poorly written. There is a lack of

technical depth. The issue addressed by the authors are trivial and
there is nothing novel in their approach. Also, the simulation
results are unconvincing since the traffic load is set below 500bps.
This paper presents a brand new approach in solving low power

requirements in analog circuits. The authors have successfully
demonstrated the performance of the new circuit through both
simulation and practical experiments. The paper itself is well organized
and readable. The work done is significant and the reviewer
recommends that this paper be accepted.
2010/4/8
REVIEWS
Outcome
15
y EiC then made a decision
Scores
and
Comments
Reject
Major Revision
Accept With
Minor Revision
Or
On Conditions
Accept
REVIEWS
Scoring & Outcome
16
y Conflicting Reviews: Accept/Reject recommendations

Reviewer #2
Reviewer #3
Accept
Accept
Reject
Likely
ACCEPT
Accept
Accept
Accept
Definitely
ACCEPT
Accept
Reject
Reject
Likely
REJECT
Reviewer #1
2010/4/8
REVIEWS
Scoring & Outcome
17
y Conflicting Reviews: Overall recommendation score

Reviewer #1
Reviewer #2
Reviewer #3
Likely
ACCEPT
ACCEPT /
REJECT
Likely
REJECT
REVIEWS
About Reviewers
18
y They are humans too!
You
Others
y They are also your competitors

y They may not want you to get ahead
y They may not want you to publish the same topic
before them!
y They try to find fault in your paper or accuse you of
flaws or wrong doing
y ACTION:
{
You need to stay cool and try to address the accusations and
negative comments based on facts. Hence, the need for
rebuttals.
2010/4/8
INTERNATIONAL FORUM
19
y2
Rebuttal
Rebut: to provide an answer or counter-proof
REBUTTAL
Paper Accept
20
y Rebut to provide an answer or counter-proof.

y EiC Decision: Accept
Accept
Correct
Any typos
Prepare
Camera-ready
Copy
Sign Copyright
Forms
Outcome
is usually
Accept!
Submit for
Publication
1. No need for rebuttal

2. Correct any minor typo or grammatical errors
3. Prepare final camera ready copy
10
2010/4/8
REBUTTAL
Accept with Conditions
21
y Accept on Conditions
Accept on
Conditions
Revise
according
to reviewers
comments
Fulfill
Reviewers
Conditions
Submit for 2nd

Review
Prepare REPLY TO REVIEWERS
1. Extract what reviewers want you to do.

2. Write a REPLY letter. (EXAMPLE)
3. Address each issue with an adequate answer. (EXAMPLE)
4. Revise the paper accordingly to changes recommended by reviewers.
5. Fulfill most, if not all, of the requested conditions
REBUTTAL
Major Revision
22
y Major Revision
Major
Revision
Perform
more
Simulations
+ research if
needed
Revise
according
to
reviewers
comments
Usually
paper
needs to
be
rewritten
Submit
for
2nd
Review
Prepare REPLY TO
REVIEWERS
1. Write a REPLY addressing all changes to the paper
2. Need to think carefully if you want to do a major revision
3. There is no guarantee of acceptance after a major revision
Outcome
is still
uncertain!
Can be
Accept or
Reject
11
2010/4/8
REBUTTAL
Reply to reviewers
23
y Example
of
Rebuttal
REBUTTAL
You versus Reviewers
24
y What if reviewers are wrong?
Ask &
Check with
Your
Professor!
Reviewer
is still
correct?
Prepare a
REPLY to
reviewers to
explain why they
are wrong and
you are right!
Submit to EiC and to

Reviewers
Check with another

3rd person!
y If reviewers accept your clarifications, then matter is
solved. Your paper will be accepted. If not, EiC can still

reject your paper!
12
2010/4/8
REBUTTAL
Psychology
25
y Avoid vending your anger at reviewers!

y EXAMPLES:
{ Reviewer 1 seems jealous of our results and do not believe in
our performance gain.
{ Reviewer 2 is stupid that he cannot understand our algorithm.
{ Reviewer 3 did not read our section 2 at all! His remarks are
childish and untrue.
{ Reviewer 2 should read our paper again!
{ Equation 5 is so simple. Why Reviewer 1 cannot understand it?
{ Reviewer 2 is unreasonable to ask us to do so many things.
{ Reviewer 1 has not read our earlier paper.
REBUTTAL
Psychology
26
y Avoid creating a suspicious atmosphere!

y It does not help to get your paper accepted.
y If reviewers misbehave, report to EiC.
y Let EiC makes the final decision after rebuttal.
y If EiC misbehaves, then it is time to withdraw your
paper and submit it elsewhere!

y EXAMPLES:
{
{
{
{
We think Reviewer 2 is our competitor and enemy.

We think Reviewer 2 is dishonest, biased, and unjust
Reviewer 1 is trying to delay the publication of our work
Reviewer 3 is envy of our success
13
2010/4/8
REBUTTAL
Psychology
27
y Reviewer is fussy:
{ Try to fulfill his/her requests as much as possible
{ i.e., keep him/her happy, interested, and positive
y Reviewer is mistaken:
{ Try to clarify his/her doubts in your reply
{ i.e., make him/her feel positive about your work
y Reviewer is unreasonable:
{ Keep your cool, try to answer his/her negative comments
{ Seek help from Editor-in-chief
{ Decide to stay in the process or quit (i.e. withdraw)
REBUTTAL
Reviewers CONDITIONS
28
y You did not cite my papers
This is unreasonable
To get your paper accept, you must cite my
papers first bad behavior!
y Your figures are not clear
Redraw or make it larger and sharper
y You need to do more experiments / simulations

This may or may not be reasonable
Some experiments are hard to repeat
Time consuming and manpower needed to redo experiments
y You need a native writer to rewrite your paper!
English aspects (reasonable)
14
2010/4/8
INTERNATIONAL FORUM
29
y3
Revision
Revise: to look over again in order to correct or improve
REVISION
(1st, 2nd, 3rd )
30
y 1st REVISION (For Journals)

{
{
{
{
{
Rewrite parts of paper

Resubmit revised paper to EiC
Letter to EiC highlighting corrections done
Your revised paper will be reviewed again!
Wait for another 2-3 months; Outcome from EiC
y 2nd Revision
{
{
{
If reviewers are still unhappy with 1st revision, EiC may ask you to do a
2nd revision
This is the last chance!!
If reviewers accept your 2nd revision, then EiC usually recommends
accept.
y 3rd Revision (?) Usually no!

15
2010/4/8
REVISION
IMPLICATIONS
31
y PAPER ACCEPTANCE & REJECT

CONTRIBUTING
FACTORS
Characteristics of good research
and a well-written paper.
1. High reviewers scores

2. Positive comments
3. Paper is well written
Paper likely to be accepted

but imperfect in some ways;
Inadequacies can be rectified.
4. Minor revision
5. Major revision
6. REJECT:
Negative comments
Low scores
Paper regarded as inadequate;

A polite form of saying reject
Paper regarded as inadequate;
A direct form of saying no way
REVISION
WHAT IF?
32
time
Time can kill you!
1st
Submission
1st
Reject
2nd
Submission
2nd
Reject
3rd
Submission
3rd
Reject
Consult with your professor; You only have 3 years or more to graduate
If after 2 or 3 resubmissions and your paper cannot be accepted in any
journals, you need to be realistic to accept that there is no appreciation
and merit of your paper.
You must abandon and move on !! Dont be stubborn.
Do not take failure as forever.
Do something you are good at.
16
2010/4/8
PAPER ACCEPTANCE
FORMULA?
33
Write your
paper well
Pray that your

3 anonymous
reviewers will
like your work
Provide
Profound
Solutions
Submit it to
the right place
for
consideration
Dont vend
your emotions
at the EiC or
reviewers
Wait and see how others

felt about your work!!
Do Quality
Research
Get your paper accepted

and published!
y So how to get your paper accepted?
PS: I will talk more about World Class High Impact Research in another seminar.
DIFFERENCES
Conferences vs. Journals
34
FACTORS
CONFERENCES
JOURNAL
Reviewing
Period
Usually 3 months
Usually 6 months to 12
months or more
Number of
Reviewers
Usually 3; Sometimes 2;
sometimes 4 or more
Usually 3
Critical
Evaluation
Metrics
Novelty; Significance;
Technical Correctness;
Relevance;
Novelty; Significance;
Impact; Technical
Correctness; etc
Usually no
Usually yes
Technical Program
Committee Chair or Cochair
Editor-in-Chief
Rebuttal
allowed?
Who decide
to accept or
reject?
17
2010/4/8
INTERNATIONAL FORUM
35
y4
Research Ethics
Ethics: a set of moral principles or good conduct
RESEARCH ETHICS
BASICS & REALITY
36
y ARE THERE BAD PEOPLE/REVIEWERS OUT THERE?

{
{
SURE!
GOOD & EVIL ARE IN BATTLE EVERYDAY !!
y MAINTAIN HIGH LEVEL OF INTEGRITY,
SEE
NO
EVIL
HONESTY, & PROFESSIONALISM

{
{
BE HONEST WITH YOUR RESEARCH RESULTS

ACCEPT & DEFEND THE TRUTH!
y BAD PRACTICES:
{
{
{
{
COPY PAPER CONTENTS FROM OTHERS!

FAKE OR COPY RESULTS
LIE ABOUT THE TRUTH
USE BULLY OR INTIMIDATION
HEAR
NO
EVIL
SAY
NO
EVIL
18
2010/4/8
RESEARCH ETHICS
WHY WE CARE?
37
y IF I LIE.. WHAT HAPPENS? (EXAMPLE)

{ SAY YOU AND OTHERS ARE A TEAM THAT INVENTED A
NEW MATERIAL TO BE USED ON A PLANE.
{ YOU ALL LIED THAT THE MATERIAL CAN BE USED FOR
THE BODY OF THE PLANE UNDER EXTREME
TURBULENCE & STRESS.
{ PLANE WAS MADE WITH YOUR PROPOSED MATERIAL
{ PLANE WENT UP TO 35,000 feet
{ PLANE EXPLODED IN THE SKY
{ 350 PEOPLE KILLED !
{ HOW DO YOU FEEL??
{ ARE YOU A MURDERER?
EXAMPLE
RESEARCH ETHICS
GOOD OR EVIL: YOU CHOOSE
38
y Great power comes with great responsibility!

y You are defenders of the truth and for good of mankind
19
2010/4/8
RESEARCH ETHICS
Your PhD is your Power! Be real!
39
y EXAMPLES
UNIVERSITY
HARVARD
UNIVERSITY
CAMBRIDGE
UNIVERSITY
Requirements for
PhD Degree
Significant & Original Research
Significant contribution to learning
-for example through the discovery of

new knowledge, the connection of
previously unrelated facts, the
development of a new theory, or the
revision of older views
http://www.admin.cam.ac.uk/univ/gsprospectus/study/research/phd.html
RESEARCH ETHICS
WHY & WHO ENFORCES THEM?
40
y Your conduct will determine:

{ Your reputation
{ Your success
{ Influence the behavior of your students
Please!
y Many organizations enforces ethics:

{ Your University
{ Your Societies: IEEE, ACM, IET, etc
{ Your Academy: e.g. Academia Sinica
{ Your Government: Ministry of Education / Ministry of Trade /
Ministry of Technology / Ministry of Law
20
2010/4/8
RESEARCH ETHICS
PEOPLE WHO CHEATED!
41
EXAMPLE (2002)
y BELL LABS RESEARCHER CHEATED & FIRED!
http://en.wikipedia.org/wiki/Jan_Hendrik_Sch%C3%B6n
Born 1970
German Physicist;
Nanotechnology &
Condensed matter
Physics; PhD from
Germany; His PhD was
subsequently removed
by his university!!
RESEARCH ETHICS
PEOPLE WHO CHEATED!
42
y MORE
EXAMPLES
Luk Van Parijs

Born 1970
Belgium
PhD Harvard
Professor at MIT
21
2010/4/8
RESEARCH ETHICS
PEOPLE WHO CHEATED!
43
y Apology for cheating (real case in 2001)
RESEARCH ETHICS
PEOPLE WHO CHEATED!
44
y STUDENTS CHEAT TOO
22
2010/4/8
JUNK CONFERENCES
FAKE PAPERS
45
EXAMPLE
http://pdos.csail.mit.edu/scigen/
y MIT created SCIgen program can generate paper with fake content + results
y Paper that doesnt make sense, but still accepted at a conference!!!
A prankster who submitted a computer-generated research

paper to the International Conference on Computer Science
and Software Engineering discovered that not only was his fake
paper accepted - its "author" is to chair a panel.
The prankster, known only by his nickname Schlangemann
(which he used to submit the paper), created the paper using
SCIGen - the automatic CS paper generator. His nickname is
taken from a German movie called Der Schlangemann.
INTERNATIONAL FORUM
46
y5
Conclusion &
Appendices
23
2010/4/8
CONCLUSION
47
TODAY, WE HAVE COVERED:

y Reviews, Rebuttals, Revision
y Paper Acceptance what it entails:
{
{
Good quality research

A well-written paper
y Your responsibility
{ Research ethics
{ Your paper acceptance means:
Formal dissemination of your work
Paper publication is not the end!
Final success depends on how others felt about your work!!!
Example: Review Outcome
Appendix
48
Dear Dr. YYYYY,
Reviewer Comments concerning your manuscript, "An Ultra-Wideband Radio System for Vehicle
Networks," have been obtained and are copied below. Based on the attached reviewers' comments, we
will not be able to publish your manuscript in our transactions.
Thank you for submitting your manuscript to the IEEE Transactions on ZZZ.
Sincerely,
Editor-in-Chief
IEEE Transactions on ZZZ
24
2010/4/8
Appendix
49
---
Reviewer: 1
Recommendation: Reject
Comments:
(There are no comments)
Additional Questions:
Summary of Evaluation: Fair
If the paper is rejected for publication, the authors should: Regard the paper as not publishable.
Organization: 4
Clarity: 4
Length: 1
References: 3
Correctness: 4
Significance: 1
Originality: 1
Attachments:
If Survey Coverage:
Contribution: 1
Appendix
50
(Reviewer 1)
What are the contributions of this paper?
: The paper presents a system concepts and experimental results for UWB-based positioning of vehicles. While I like to see
experimental verification of results (and thus was favorably disposed to the paper when first looking at it), I find that the
originality and novelty of the paper is not sufficient for publication in a prestigious medium such as IEEE transactions. The
architecture and transmission protocol is straightforward; the location algorithm for TDOA is well-known, and the use of UWB for
ranging at distances up to 30 m has been shown with practical demonstrators multiple times. While the results appear correct, I
cannot find that they are sufficiently original to justify publication.
What are some ways in which the paper could be improved? Please supply any additional important references that you feel the
author omitted which should be noted in the paper.
:
25
2010/4/8
Appendix
51
(Reviewer: 2)
Recommendation: Reject
Comments: see review report
Additional Questions:
Summary of Evaluation: Fair
If the paper is rejected for publication, the authors should: Prepare a major revision and resubmit to The ITS Transactions as a new paper
Organization: 5
Clarity: 3
Length: 5
References: 1
Correctness: 2
Significance: 1
Originality: 1
Attachments:
If Survey Coverage:
Contribution: 2
Appendix
52
(Reviewer 2)
: see review report attached
What are some ways in which the paper could be improved? Please supply any additional
important references that you feel the author omitted which should be noted in the
paper.
: see review report attached
26
2010/4/8
Appendix
53
(Reviewer: 3)
Recommendation: Prepare A Major Revision For A New Review
Summary of Evaluation: Good
If the paper is rejected for publication, the authors should: Prepare a major revision and resubmit to The ITS
Transactions as a new paper
Organization:
3
Clarity: 4
Length: 3
References: 4
Correctness: 4
Significance: 3
Originality: 3
Attachments:
If Survey Coverage: 3
Contribution: 3
Appendix
54
(Reviewer 3)
: * Proposed a system concept of "unilateral" positioning which is different from most non-GPS positioning system
* Prosposed a transmitter synchronization scheme (TDMA) to achieve the high resolution of TDOA estimates.
* Developed a system demonstrator to show the feasibility and performance of this proposed positioning system in a specific
outdoor test scenario.
What are some ways in which the paper could be improved? Please supply any additional important references that you feel the
author omitted which should be noted in the paper.
: * Authors used a rudimental approach to solve the nonlinear TDOA equations which requires known z-coordinate and has the
singularity problem. Suggest use the two-stage least square solution proposed by Y. Chan in reference [18].
* Authors conducted some simulations to evaluate the impact of the transmitter configuration on the positioning performance and
identified a particular configuration with five transmitters. Suggest authors check the following references for a simulation work
and an analytical evaulation on a proposed four "nodes" configuration.
[1]. J. Ni, et al; UWB Tracking System Design for Free-Flyers, AIAA Space 2004 Conference and Exposition, San Diego, CA,
September 28-30, 2004.
[2]. J. Ni, et al; Design and Performance Analysis of a UWB Tracking System for Space Applications, 2005 IEEE/ACES
International Conference on Wireless Communications and Applied Computational Electromagnetics, Honolulu, Hawaii, April 37, 2005.
* The outdoor test showed sub-meter positioning accuracy of the system demonstrator in a confined area of about 35m x 25m.
Since authors claimed the UWB signal coverage is a area of a few hundred meters, it will be helpful to show the system's feasibility
and accuracy in a larger (also more closer to a practical scenario) area. To test if the positioning performance is degraded when the
coverage area is increased.
27
2010/4/8
Example: Rebuttal
Appendix
55
y REPLY to reviewers
Example: Rebuttal
Appendix
56
y REPLY
to reviewers
28
2010/4/8
Example: Rebuttal
Appendix
57
y REPLY to reviewers
29

Lecture Notes 3 PDF

Încărcat de

Informații document

Descriere originală:

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Lecture Notes 3 PDF

Încărcat de

Drepturi de autor:

Formate disponibile

2010/4/8

How to get my paper accepted?

y ESSAY WRITING CONTEST: Core students please

submit a 1-page essay with the title: My Thoughts on

y TUTORIAL HOURS: Starting from 26 April 2010,

every Monday and Wednesday. Open to EECS PhD &

Time: 2pm 3pm.

Before you submit your paper, ensure it is:

Return scores to EiC

Review period can

paper regarding various aspects,

y Reviewers are looking for (main metrics):

Did you introduce anything new in your paper?

Are your work significant or is it just marginal work?

y REVIEWER SCORING (see appendix for more examples)

Score (1: lowest; 5: highest)

y REVIEWERS COMMENTS (see appendix for more examples)

This paper is poorly organized and poorly written. There is a lack of

This paper presents a brand new approach in solving low power

y EiC then made a decision

y Conflicting Reviews: Accept/Reject recommendations

y Conflicting Reviews: Overall recommendation score

y They are humans too!

y They are also your competitors

y Rebut to provide an answer or counter-proof.

1. No need for rebuttal

Submit for 2nd

Prepare REPLY TO REVIEWERS

1. Extract what reviewers want you to do.

y What if reviewers are wrong?

Submit to EiC and to

Check with another

y If reviewers accept your clarifications, then matter is

solved. Your paper will be accepted. If not, EiC can still

y Avoid vending your anger at reviewers!

y Avoid creating a suspicious atmosphere!

paper and submit it elsewhere!

We think Reviewer 2 is our competitor and enemy.

y You did not cite my papers

y Your figures are not clear

Redraw or make it larger and sharper

y You need to do more experiments / simulations

y You need a native writer to rewrite your paper!

English aspects (reasonable)

y 1st REVISION (For Journals)

Rewrite parts of paper

y 3rd Revision (?) Usually no!

y PAPER ACCEPTANCE & REJECT

1. High reviewers scores

Paper likely to be accepted

Paper regarded as inadequate;

Time can kill you!

Pray that your

Wait and see how others

Get your paper accepted

y So how to get your paper accepted?

y ARE THERE BAD PEOPLE/REVIEWERS OUT THERE?

y MAINTAIN HIGH LEVEL OF INTEGRITY,

HONESTY, & PROFESSIONALISM

BE HONEST WITH YOUR RESEARCH RESULTS

COPY PAPER CONTENTS FROM OTHERS!

y IF I LIE.. WHAT HAPPENS? (EXAMPLE)