News

    Please join our mailing list to participate in discussions regarding the shared task

  • 9 February 2017: Annotated test data is now available and results for the three scenarios, including a break-down by task. Congratulations to the winners!
  • 10 January 2016: ScienceIE competition phase starts. Test your systems on the dev and test data and submit your results at CodaLab.
  • 6 January 2016: CodaLab competition is up and running for ScienceIE submissions. Access it here
  • 12 October 2016: We removed inconsistencies in the training data. The new version is available for download.
  • 13 September 2016: Minor update to evaluation script to only measure performance for Subtask A
  • 12 September 2016: Submissions will be managed through CodaLab!
  • 10 September 2016: Registration!
  • 5 September 2016: Training data now available for download!
  • 1 August 2016: Development data, evaluation and utility scripts now available for download!
  • 14 July 2016: ScienceIE website online!

    • Overview

    The shared task ScienceIE at SemEval 2017 deals with automatic extraction of keyphrases from Computer Science, Material Sciences and Physics publications, as well as extracting types of keyphrases and relations between keyphrases.
    PROCESS, TASK and MATERIAL form the fundamental objects in scientific works. Scientific research and practice is founded upon gaining, maintaining and understanding the body of existing scientific work in specific areas related to such fundamental objects. Some typical questions, researchers and practitioners more than often face are:

    • which papers have addressed a specific TASK ?
    • which papers have studied a PROCESS or variants ?
    • which papers have utilized such MATERIALS ?
    • which papers have addressed this TASK using variants of this PROCESS ?

      • Review papers are seldomly available in most research areas, and ability of search engines for scientific publications is limited. In addition to this, researchers often only have a vague search requirements which makes it hard to answer the above questions efficiently.

      Automatically extracting keyphrases of the scientific documents, then labelling them and extracting relationships between them can address the above questions efficiently. This will further provide ultilities that can recommend relevant article to readers, match reviewers to submissions and help to explore huge collections of papers.