Bachelor Project Computer Science

Benchmark and Comparison of State-of-the-Art Ontology and Vocabulary Repositories for Social Sciences and Humanities

Abstract

The increasing adoption of the Semantic Web in the Social Sciences and Hu- manities (SSH) has led to the development of numerous ontology and vocabu- lary repositories. These repositories serve as crucial resources for structuring, sharing, and reusing domain  knowledge.  This work provides a benchmark and comparative analysis of leading repositories, evaluating their scope, ac- cessibility, interoperability, and usability.  By analyzing platforms such as the Ontology Lookup Service (OLS), BioPortal, the Social Science Thesaurus, and other domain-specific repositories, we assess their relevance for SSH research. The purpose of the study is to guide students and researchers in selecting the most appropriate repository for their work.  Additionally, a practical im-plementation  proposal  for a bachelor’s  dissertation is outlined, focusing  on ontology evaluation and integration within an SSH research framework.

1 Introduction

The Semantic Web has significantly influenced knowledge management and data inte- gration in various disciplines, including Social Sciences and Humanities[3].   The use of ontologies and controlled vocabularies facilitates semantic interoperability, making repos- itories essential tools for researchers.  However, with numerous available repositories, a comparative analysis is necessary to determine the most suitable for SSH applications[10].

Key Terms:

• Semantic Web

• Benchmark

• Knowledge Management

•  Ontology repository

•  Social Sciences and Humanities (SSH)

2 Background

With the increasing usage of the Sematic Web in the Social Sciences and Humanities (SSH), ontology and vocabulary repositories developed rapidly.  These ontologies and repositories are important resources for structuring, sharing, and reusing domain knowledge, enabling researchers to work with well-organized and machine-readable data.   However, there is limited guidance for evaluating these repositories in terms of scope, accessibility, inter- operability, and usability, especially in the field of SSH. The comparison and benchmark analysis of the ontology and repositories can help researchers navigate and apply these tools effectively in SSH contexts.

• Benchmark: A systematic comparison and evaluation of tools or systems based on defined criteria to determine their performance or suitability

• Ontology:  A structured representation of knowledge within a domain,  defining concepts and the relationships between them to enable semantic understanding.

• Vocabulary Repositories: Platforms that store and provide access to controlled vocabularies or ontologies, facilitating data organization, retrieval, and reuse.

• Social Sciences and Humanities:Academic disciplines focused on human society, behavior, history, language, and culture, often involving qualitative or complex data.

3 Problem

Ontology and vocabulary repositories are important for managing knowledge in the field of Social Sciences and Humanities (SSH). However, a great number of these resources were initially designed for structured, scientific domains; They are not fully suitable to specific need from SSH research. Therefore, researchers working in the SSH field will always face the difficulties in deciding which repositories are the most suitable,  particularly when considering the subject’s scope and coverage, interoperability, usability and integration with research tools such as RDF and SPARQL.

In addition, integration of multiple ontologies, using RDF and SPARQL, is necessary for SSH because existing repositories might lack structural flexibility and user-friendly design.  Integration is important because it can significantly improve the general scope and coverage, semantic consistency, and interoperability of ontological resources used in SSH research.

This research will solve these problems by providing comparisons of ontology and vo- cabulary repositories,  assess key evaluation criteria,  and provide  SSH researchers with practical guidance.

4    Related Work

The paper [8] presents the evaluations of scope, accessibility, interoperability, and usability to support a benchmark and comparative analysis of ontology and vocabulary repositories, hence improving the semantic consistency, interoperability, and usability and accessibility in the field of Social Sciences and Humanities.

This paper[11] presents the evaluations of coverage  and completeness,  usability  and accessibility, and interoperability, offering a structured benchmark of ontology and vo- cabulary repositories to support selection of semantic resources in Social Sciences and Humanities research.

This paper[2] directly provides a structured benchmark and comparative analysis of ontology and vocabulary repositories, evaluating their coverage, usability, interoperability, and relevance for supporting Semantic Web applications in Social Sciences and Humanities research.

This paper[1] relates to the topic by showing how ontologies can be used to organize and present SSH knowledge clearly, which is also helpful to explain why good ontology repositories are important for education and research.

This paper[13] shows the importance of interoperable infrastructures and FAIR data principles in SSH, supporting the need to benchmark ontology and vocabulary repositories to improve interoperability and usability.

5 Research Question(s)

•  How do leading ontology repositories compare in terms of scope and coverage, in- teroperability, usability, community support, and integration with research tools for Social Sciences and Humanities (SSH) research ?

•  How can multiple ontologies be integrated into an SSH research framework using RDF and SPARQL to enhance knowledge management and data integration?

• What improvements can be made to the structure and usability of existing SSH vocabulary repositories to better serve the needs of researchers?

6 Approach

This research follows a structured four-step methodology to benchmark and compare lead- ing ontology and vocabulary repositories for SSH research:

1. Ontology and Vocabulary Repositories Selection: Ontology and vocabulary repositories provide structured knowledge representations that enhance data discov- ery and integration. The most notable repositories were chosen:

• Ontology Lookup Service (OLS) – A service aggregating ontologies across multiple domains[4].

• BioPortal –  Originally focused on biomedical ontologies but expanding to social sciences[9].

• LOV (Linked Open Vocabularies) – A repository for linked data vocabularies[12].

• Social Science Thesaurus – A specialized vocabulary for social science research[5].

• BARTOC (Basic Register of Thesauri, Ontologies & Classifications)

– A catalog of knowledge organization systems[7].

• CLARIAH FAIR Vocabulary Registry – A registry of vocabularies from domains used in the Social Sciences and Humanities (SSH)

6.1    Evaluation Based on Benchmarking Criteria:

To evaluate these repositories, the following criteria are considered:

• Scope and Coverage – The breadth of subjects covered within SSH.

• Interoperability – Compatibility with linked data and Semantic Web technologies[6]

• Usability – The user interface and ease of access for non-technical researchers.

• Community Support and Maintenance – Frequency of updates and com- munity engagement.

• Integration with Research Tools – Compatibility with RDF, SPARQL, and data visualization tools.

Table 1: Evaluation Matrix

6.1.1    LOV (Linked Open Vocabularies)

• Coverage (SSH): Medium – LOV includes several relevant vocabularies for SSH, but it lacks deep domain-specific coverage.

• Interoperability: High – It fully supports linked data principles and uses standard formats like RDF and OWL.

• Usability: High – The platform. is user-friendly, with a clean interface and clear navigation.

• Sustainability:  High – It is well-maintained with regular updates and long- term availability.

• Community Engagement: High – Active user contributions and documen- tation reflect strong community support.

• Multilingual: Low – Most vocabularies are only available in English, limiting multilingual support.

6.1.2 BioPortal

• Coverage (SSH): Low – BioPortal is focused primarily on biomedical ontolo- gies thus they have limited SSH-relevant content.

• Interoperability: High – It uses robust ontology standards and provides API and SPARQL access.

• Usability: High – The interface is highly user-friendly and includes visual- ization tools for browsing ontologies.

• Sustainability:  High – It is supported by Stanford thus it shows long-term institutional support and will get regular updates.

• Community Engagement: Medium – There is moderate user feedback and activity, but not SSH-specific.

• Multilingual: Low – Most ontologies are monolingual and there is no function to translate content to other languages.

6.1.3 FAIRsharing

• Coverage (SSH): Medium –  FAIRsharing  includes  some  SSH-related re- sources, but they are not its primary focus.

• Interoperability: Medium – It provides metadata standards but lacks deeper semantic integration options.

• Usability: Medium – The platform. is usable but not especially optimized for SSH researchers.

• Sustainability:  High – Maintained as part of FAIR initiatives, and it regu- larly gets updated.

• Community  Engagement:   Medium – Some user interaction exists, but active contributions are limited.

• Multilingual: Medium – Some support exists for multilingual access, but it is not applied to all content.

6.1.4 VocBench

• Coverage (SSH): High – VocBench supports the creation of SSH-specific vocab- ularies and ontologies.

• Interoperability: High – It supports SKOS, OWL, RDF, and integrates well with external semantic tools.

• Usability: Medium – It is powerful but it is difficult for non-technical users to use.

• Sustainability: Medium – Development is ongoing but depends on specific projects or institutions.

• Community Engagement: High – There is strong involvement from open-source and many academic communities.

• Multilingual:  High – Full support for multilingual vocabularies is built into the tool.

6.1.5    SSHOC Vocabulary Commons

• Coverage (SSH): High – Specifically designed to serve SSH domains with relevant vocabularies.

• Interoperability:   Medium – For RDF-based,  some vocabularies lack detailed alignment with external ontologies.

• Usability: Medium – The interface is functional but could be more user friendly.

• Sustainability: Medium – Continued development depends on project funding and EU infrastructure.

• Community Engagement: High – Community contributions and involvement in development are actively encouraged.

• Multilingual: High – Many vocabularies are available in multiple languages, sup- porting multilingual use cases.

6.2     Integration of Ontologies by Using RDF and  SPARQL: RDF and SPARQL will be used to integrate selected ontologies from different repositories, enabling cross-domain alignment and semantic linking within the context of SSH research.

6.3 Comparative Analysis: Each repository is evaluated against the above criteria, highlighting strengths and weaknesses. For example, while LOV excels in linked data integration, BioPortal offers robust ontology management tools, but is less SSH-focused. The Social Science Thesaurus provides rich domain-specific terminologies, but has limited interoperability features.  This analysis aims to provide practical recommendations to help SSH researchers select the most suitable repositories for their needs.

7 Plan

The research plan is structured around six key phases:

1. Ontology and Vocabulary Repositories Selection: Identify and select repre- sentative repositories that cover diversity and are relevant to SSH research.

2. Evaluation based on Benchmarking Criteria: Assess the evaluations of ontol- ogy and vocabulary repositories in SSH is based on several key criteria:  Coverage and Completeness; Semantic Consistency; Usability and Accessibility; Interoperability; Maintainability and Sustainability; Domain Specificity; Community Engagement.

3. Achieve Integration: Apply RDF and SPARQL tools to combine ontologies from different repositories, allowing cross-domain alignment and semantic linking within the context of SSH research.

4. Comparative Analysis: Each  repository  is  evaluated  against the  above  crite- ria, highlighting strengths and weaknesses, highlighting each repository’s strengths, weaknesses, and how it fits for SSH contexts

5. Reporting and presentation: Summarize the findings, accomplish the thesis re- port and hand in.

8 Conclusion

The selection of an appropriate ontology repository is crucial for SSH research.   This study benchmarks the leading repositories, offering insight into their suitability. For stu- dents, practical projects in ontology evaluation and integration provide valuable hands-on experience in Semantic Web applications.

References

[1] V. Atamanchuk and P. Atamanchuk.   Ontological modeling in humanities.  In In- ternational  Scientific-Practical  Conference   ”Information  Technology for  Education, Science and  Technics”. Springer Nature Switzerland, 2022.

[2] K. Baclawski and T. Schneider.  The open ontology repository initiative:  Require- ments and research challenges.  In Proceedings  of  Workshop  on   Collaborative  Con- struction, Management and Linking of Structured Knowledge  at  the ISWC, 2009.

[3]  T. Berners-Lee,  J. Hendler,  and  O.  Lassila.   Web  semantic.   Scientific  American, 284(5):34—43, 2001.

[4]  R. G. Cˆot´e, P. Jones, R. Apweiler, and H. Hermjakob. The ontology lookup service.

BMC Bioinformatics, 2006.

[5]  GESIS.        Social   science   thesaurus. https://www.gesis.org/en/research/ thesaurus, 2020.

[6]  T. Heath and C. Bizer.  Linked Data:  Evolving  the  Web  into  a  Global  Data  Space.

Morgan & Claypool, 2011.

[7] A. Kempf et al. Bartoc: A registry of knowledge organization systems.  International Journal on Digital Libraries, 2019.

[8] K. Meijer, K. H. Cluster, and M. Windhouwer. The clariah fair vocabulary registry.

In CLARIN Annual  Conference Proceedings, page 158, 2024.

[9]  M. A. Musen et al.  Bioportal:  Ontologies and integrated data resources.   Nucleic Acids Research, 2012.

[10] A. Name. Title of the article.  Journal Name, Volume:Pages, Year.

[11]  The Hyve. Evaluation of fair data assessment tools, 2022.  Accessed:  2024-04-15.

[12]  P. Vandenbussche  et  al.   Linked  open  vocabularies  (lov):   A  gateway  to  reusable semantic web vocabularies.  Semantic  Web  Journal, 2017.

[13]  I. I. Verˇsi´c and J. Ausserhofer. Social sciences, humanities and their interoperability with the european open science cloud: What is sshoc?  Mitteilungen Der  Vereinigung Osterreichischer Bibliothekarinnen  Und  Bibliothekare, 72(2):383—391, 2019.