Cosmos QA : Machine Reading Comprehension with Contextual Commonsense Reasoning (EMNLP'2019)
Paper » Code » Dataset »What is Cosmos QA?
Cosmos QA is a large-scale dataset of 35.6K problems that require commonsense-based reading comprehension, formulated as multiple-choice questions. It focuses on reading between the lines over a diverse collection of people's everyday narratives, asking questions concerning on the likely causes or effects of events that require reasoning beyond the exact text spans in the context.
Reading comprehension requires not only understanding what is stated explicitly in text, but also reading between the lines, i.e., understanding what is not stated yet obviously true (Norvig, 1987).
Cosmos QA Leaderboard
To benchmark approaches to Cosmos QA, we have a leaderboard for the test set. If you have a model for solving Cosmos QA and would like to make a submission, you should follow the rules and policies, and create your submission here.
Cosmos QA Examples
Example 1
Paragraph: It's a very humbling experience when you need someone to dress you every morning, tie your shoes, and put your hair up. Every menial task takes an unprecedented amount of effort. It made me appreciate Dan even more. But anyway I shan't dwell on this (I'm not dying after all) and not let it detact from my lovely 5 days with my friends visiting from Jersey.
Question: What's a possible reason the writer needed someone to dress him every morning?
Options: (click the choice to see if it's correct or not)
Example 2
Paragraph: A woman had topped herself by jumping off the roof of the hospital she had just recently been admitted to. She was there because the first or perhaps latest suicide attempt was unsuccessful. She put her clothes on, folded the hospital gown and made the bed. She walked through the unit unimpeded and took the elevator to the top floor.
Question: What would have happened to the woman if the staff at the hospital were doing their job properly?
Options: (click the choice to see if it's correct or not)
