We used strictly logical reasoning in our early look at this space. This approach is well worth a look, links to paper and publication below. Technical.
Microsoft Research Blog
Microsoft LReasoner leads the ReClor challenge on logical reasoning
Published June 24, 2021, By Natural Language Computing Group
For many years AI researchers have sought to build upon traditional machine learning, which trains technology to process facts and learn from them, and develop machine reasoning, in which programs apply logic to data and solve problems – comparable to the way humans think. For a system to analyze multiple sets of logical arguments, it requires both critical thinking and combinatorial reasoning abilities.
PUBLICATION
ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning
One of the current benchmarks for evaluating a system’s logical reasoning ability is ReClor, a Reading Comprehension Dataset Requiring Logical Reasoning. ReClor is a dataset built from logical reasoning problems used in standardized admission tests, including the Law School Admission Test (LSAT) and Graduate Management Admission Test (GMAT).
Today, we are excited to announce that Microsoft’s LReasoner system is the top-rated performer on the official ReCLor leaderboard. LReasoner also significantly exceeded human performance, as measured by the average accuracy of 10 college students who each answered 10 randomly selected test questions and reported in the ReClor paper. .... and publication.
No comments:
Post a Comment