Analysis of inter-transcriber consistency in the Cat-ToBI prosodic labeling system

David Escudero; Lourdes Aguilar; Maria Del Mar Vanrell; Pilar Prieto

doi:10.1016/j.specom.2011.12.002

Analysis of inter-transcriber consistency in the Cat-ToBI prosodic labeling system

David Escudero, Lourdes Aguilar, Maria Del Mar Vanrell, Pilar Prieto

Research output: Contribution to journal › Article › Research › peer-review

20 Citations (Scopus)

Abstract

A set of tools to analyze inconsistencies observed in a Cat-ToBI labeling experiment are presented. We formalize and use the metrics that are commonly used in inconsistency tests. The metrics are systematically applied to analyze the robustness of every symbol and every pair of transcribers. The results reveal agreement rates for this study that are comparable to previous ToBI inter-reliability tests. The inter-transcriber confusion rates are transformed into distance matrices to use multidimensional scaling for visualizing the confusion between the different ToBI symbols and the disagreement between the raters. Potential different labeling criteria are identified and subsets of symbols that are candidates to be fused are proposed. © 2011 Elsevier B.V. All rights reserved.

Original language	English
Pages (from-to)	566-582
Journal	Speech Communication
Volume	54
Issue number	4
DOIs	https://doi.org/10.1016/j.specom.2011.12.002
Publication status	Published - 1 Jan 2012

Keywords

Inter-transcriber consistency
Prosodic labeling
Prosody
ToBI

Access to Document

10.1016/j.specom.2011.12.002

Cite this

@article{adea29624d32419db425878755e0cda4,

title = "Analysis of inter-transcriber consistency in the Cat-ToBI prosodic labeling system",

abstract = "A set of tools to analyze inconsistencies observed in a Cat-ToBI labeling experiment are presented. We formalize and use the metrics that are commonly used in inconsistency tests. The metrics are systematically applied to analyze the robustness of every symbol and every pair of transcribers. The results reveal agreement rates for this study that are comparable to previous ToBI inter-reliability tests. The inter-transcriber confusion rates are transformed into distance matrices to use multidimensional scaling for visualizing the confusion between the different ToBI symbols and the disagreement between the raters. Potential different labeling criteria are identified and subsets of symbols that are candidates to be fused are proposed. {\textcopyright} 2011 Elsevier B.V. All rights reserved.",

keywords = "Inter-transcriber consistency, Prosodic labeling, Prosody, ToBI",

author = "David Escudero and Lourdes Aguilar and Vanrell, {Maria Del Mar} and Pilar Prieto",

year = "2012",

month = jan,

day = "1",

doi = "10.1016/j.specom.2011.12.002",

language = "English",

volume = "54",

pages = "566--582",

journal = "Speech Communication",

issn = "0167-6393",

number = "4",

}

TY - JOUR

T1 - Analysis of inter-transcriber consistency in the Cat-ToBI prosodic labeling system

AU - Escudero, David

AU - Aguilar, Lourdes

AU - Vanrell, Maria Del Mar

AU - Prieto, Pilar

PY - 2012/1/1

Y1 - 2012/1/1

N2 - A set of tools to analyze inconsistencies observed in a Cat-ToBI labeling experiment are presented. We formalize and use the metrics that are commonly used in inconsistency tests. The metrics are systematically applied to analyze the robustness of every symbol and every pair of transcribers. The results reveal agreement rates for this study that are comparable to previous ToBI inter-reliability tests. The inter-transcriber confusion rates are transformed into distance matrices to use multidimensional scaling for visualizing the confusion between the different ToBI symbols and the disagreement between the raters. Potential different labeling criteria are identified and subsets of symbols that are candidates to be fused are proposed. © 2011 Elsevier B.V. All rights reserved.

AB - A set of tools to analyze inconsistencies observed in a Cat-ToBI labeling experiment are presented. We formalize and use the metrics that are commonly used in inconsistency tests. The metrics are systematically applied to analyze the robustness of every symbol and every pair of transcribers. The results reveal agreement rates for this study that are comparable to previous ToBI inter-reliability tests. The inter-transcriber confusion rates are transformed into distance matrices to use multidimensional scaling for visualizing the confusion between the different ToBI symbols and the disagreement between the raters. Potential different labeling criteria are identified and subsets of symbols that are candidates to be fused are proposed. © 2011 Elsevier B.V. All rights reserved.

KW - Inter-transcriber consistency

KW - Prosodic labeling

KW - Prosody

KW - ToBI

U2 - 10.1016/j.specom.2011.12.002

DO - 10.1016/j.specom.2011.12.002

M3 - Article

SN - 0167-6393

VL - 54

SP - 566

EP - 582

JO - Speech Communication

JF - Speech Communication

IS - 4

ER -

Analysis of inter-transcriber consistency in the Cat-ToBI prosodic labeling system

Abstract

Keywords

Access to Document

Fingerprint

Cite this