A multilayer corpus for research on discourse models

  • Latest stable version: 7.0.0
  • Latest commit:  
GUM corpus visualizations

GUM is an open source multilayer corpus of richly annotated texts from twelve text types. Annotations include:

  • Multiple POS tags, morphological features and lemmatization
  • Sentence segmentation and rough speech act
  • Document structure in TEI XML (paragraphs, headings, figures, etc.)
  • ISO date/time annotations
  • Speaker and addressee information (where relevant)
  • Constituent and dependency syntax
  • Information status (given, accessible, new, split antecedent)
  • Entity and coreference annotation, including bridging anaphora
  • Entity linking (Wikification)
  • Discourse parses in Rhetorical Structure Theory and discourse dependencies

The corpus is collected and expanded by students as part of the curriculum in LING-367 Computational Corpus Linguistics at Georgetown University. Each year students begin by choosing a text from within one of four possible genres, and as we learn about different annotation types and standards, participants are responsible for analyzing their own document, to which they add more and more layers of analysis: from part-of-speech tagging, through treebanking, entity recognition, discourse parsing, and more. Texts are chosen from openly available sources, and students who wish to contribute their analyses at the end of semester can do so under a Creative Commons license. The resulting data is checked for consistency and published online via GitHub. See this page for a list of contributors.

Text types and sources

Genre, modality, intended recipients, background knowledge and communicative intent all influence how we use language extensively. The selection of text types in GUM is meant to represent different communicative settings, while coming from sources that are readily and openly available, so that new texts can be annotated and published with ease, without restrictive licenses and free of charge. In order to support a collaborative environment, each year we work on texts from four genres, creating small groups of students conducting research on one type of texts, which can be compared with three others within the classroom. Every three year, we change genres and select four new types of data to work on. The GUM corpus currently contains the following proportions of texts:

Text typeSourceDocumentsTokens
News storiesWikinews2114,094
Travel guidesWikivoyage1714,967
How-to guideswikiHow1916,927
Academic writingVarious1615,112
Forum discussionsreddit1816,286
ConversationsUCSB Corpus55,701
Political speechesVarious54,834
CC VlogsYouTube55,189