Subjectivity in Clone Judgment: Can We Ever Agree?

An objective definition of what a code clone is currently eludes the field.\r\nA small study was performed at an international workshop to elicit\r\njudgments and discussions from world experts regarding what characteristics\r\ndefine a code clone. Less than half of the clone candidates judged had\r\n80% agreement amongst the judges. Judges appeared to differ primarily in\r\ntheir criteria for judgment rather than their interpretation of the clone\r\ncandidates. In subsequent open discussion the judges provided several\r\nreasons for their judgments. The study casts additional doubt on the\r\nreliability of experimental results in the field when the full criterion\r\nfor clone judgment is not spelled out.