Annotation consensus: implications for passage recommendation in scientific literature.

ABSTRACTWe present a study of the degree to which annotations overlap when several researchers read the same set of scientific articles. Our objective is to determine whether there is sufficient evidence to suggest that information about which passages initial readers tend to annotate might be used to recommend important passages to later readers of the same material. We found that readers exhibit a high degree of overlap in the passages they annotate, that these passages account for a small but significant fraction of the total document, and that such passages are distributed throughout a document rather than concentrated in the same few sections in each paper (e.g., the results section). These findings indicate that work on developing a passage recommendation model based on annotation is warranted.