Evaluation of multi-terminology super-concepts for information retrieval.

Background: Following a recent change in the indexing policy for French quality controlled health gateway CISMeF, multiple terminologies are now being used for indexing in addition to MeSH (R). Objective: To evaluate precision and recall of super-concepts for information retrieval in a multi-terminology paradigm compared to MeSH-only. Methods: We evaluate the relevance of resources retrieved by multi-terminology super-concepts and MeSH-only super-concepts queries. Results: Recall was 8-14% higher for multi-terminology super-concepts compared to MeSH only super-concepts. Precision decreased from 0.66 for MeSH only super-concepts to 0.61 for multi-terminology super-concepts. Retrieval performance was found to vary significantly depending on the super-concepts (p<10(-4)) and indexing methods (manual vs automatic; p<0.004). Conclusion: A multi-terminology paradigm contributes to increase recall but lowers precision. Automated tools for indexing are not accurate enough to allow a very precise information retrieval.