Overview

A multilayer corpus for research on discourse models

  • Latest stable version: 7.0.0
  • Latest commit:  
GUM corpus visualizations

GUM is an open source multilayer corpus of richly annotated texts from 16 text types. Annotations include:

  • Multiple POS tags, morphological features, morphological segmentation and lemmatization
  • Sentence segmentation and rough speech act
  • Document structure in TEI XML (paragraphs, headings, figures, etc.)
  • Normalized ISO date/time annotations
  • Speaker and addressee information (where relevant)
  • Constituent and (enhanced) Universal Dependencies syntax
  • Select grammatical constructions using Construction Grammar (CxG)
  • Information status (given, accessible, new, split antecedent) and salience
  • Entity and coreference annotation, including bridging anaphora
  • Entity linking (Wikification)
  • Discourse parses in enhanced Rhetorical Structure Theory (eRST), including connective detection, non-projective relations and and discourse dependencies
  • Abstractive summarization

The corpus is collected and expanded by students as part of the curriculum in LING-4427 Computational Corpus Linguistics at Georgetown University. Each year students begin by choosing a text from within one of four possible genres, and as we learn about different annotation types and standards, participants are responsible for analyzing their own document, to which they add more and more layers of analysis: from part-of-speech tagging, through treebanking, entity recognition, discourse parsing, and more. Texts are chosen from openly available sources, and students who wish to contribute their analyses at the end of semester can do so under a Creative Commons license. The resulting data is checked for consistency and published online via GitHub. See this page for a list of contributors.

Text types and sources

Genre, modality, intended recipients, background knowledge and communicative intent all influence how we use language extensively. The selection of text types in GUM is meant to represent different communicative settings, while coming from sources that are readily and openly available, so that new texts can be annotated and published with ease, without restrictive licenses and free of charge. In order to support a collaborative environment, each year we work on texts from four genres, creating small groups of students conducting research on one type of texts, which can be compared with three others within the classroom. Every three years, we change genres and select four new types of data to work on. The GUM corpus currently contains the following proportions of texts:

Text typeSourceDocsTokens
Academic writingVarious1817,169
BiographiesWikipedia2018,213
CC VlogsYouTube1516,864
ConversationsUCSB Corpus1416,391
Courtroom transcriptsVarious67,069
EssaysVarious55,750
FictionVarious1917,510
Forumreddit1816,364
How-to guideswikiHow1917,081
InterviewsWikinews1918,196
LettersVarious65,982
News storiesWikinews2316,146
PodcastsVarious55,737
Political speechesVarious1516,720
TextbooksOpenStax1516,693
Travel guidesWikivoyage1816,514
Total235228,399

For more genres, such as poetry, math and esports, also see our out-of-domain GENTLE corpus.