TY - CHAP
T1 - Watching the News
T2 - Towards VideoQA Models that can Read
AU - Jahagirdar, Soumya
AU - Mathew, Minesh
AU - Karatzas, Dimosthenis
AU - Jawahar, C. V.
N1 - Funding Information:
Acknowledgements This work is supported by MeitY, Government of India.
Publisher Copyright:
© 2023 IEEE.
PY - 2023
Y1 - 2023
N2 - Video Question Answering methods focus on common-sense reasoning and visual cognition of objects or persons and their interactions over time. Current VideoQA approaches ignore the textual information present in the video. Instead, we argue that textual information is complementary to the action and provides essential contextualisation cues to the reasoning process. To this end, we propose a novel VideoQA task that requires reading and understanding the text in the video. To explore this direction, we focus on news videos and require QA systems to comprehend and answer questions about the topics presented by combining visual and textual cues in the video. We introduce the "NewsVideoQA"dataset that comprises more than 8, 600 QA pairs on 3, 000+ news videos obtained from diverse news channels from around the world. We demonstrate the limitations of current Scene Text VQA and VideoQA methods and propose ways to incorporate scene text information into VideoQA methods.
AB - Video Question Answering methods focus on common-sense reasoning and visual cognition of objects or persons and their interactions over time. Current VideoQA approaches ignore the textual information present in the video. Instead, we argue that textual information is complementary to the action and provides essential contextualisation cues to the reasoning process. To this end, we propose a novel VideoQA task that requires reading and understanding the text in the video. To explore this direction, we focus on news videos and require QA systems to comprehend and answer questions about the topics presented by combining visual and textual cues in the video. We introduce the "NewsVideoQA"dataset that comprises more than 8, 600 QA pairs on 3, 000+ news videos obtained from diverse news channels from around the world. We demonstrate the limitations of current Scene Text VQA and VideoQA methods and propose ways to incorporate scene text information into VideoQA methods.
KW - Algorithms: Vision + language and/or other modalities
KW - Arts/games/social media
KW - Video recognition and understanding (tracking, action recognition, etc.)
UR - http://www.scopus.com/inward/record.url?scp=85149048910&partnerID=8YFLogxK
UR - https://www.mendeley.com/catalogue/17f200fa-7882-347f-ad51-4d61e7bff2de/
U2 - 10.1109/WACV56688.2023.00442
DO - 10.1109/WACV56688.2023.00442
M3 - Chapter
AN - SCOPUS:85149048910
SN - 9781665493468
T3 - Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023
SP - 4430
EP - 4439
BT - WACV
PB - Institute of Electrical and Electronics Engineers Inc.
ER -