Research Article: Novel knowledge-based system with relation detection and textual evidence for question answering research

Date Published: October 3, 2018

Publisher: Public Library of Science

Author(s): Hai-Tao Zheng, Zuo-You Fu, Jin-Yuan Chen, Arun Kumar Sangaiah, Yong Jiang, Cong-Zhi Zhao, Yifan Peng.


With the development of large-scale knowledge bases (KBs), knowledge-based question answering (KBQA) has become an important research topic in recent years. The key task in KBQA is relation detection, which is the process of finding a compatible answer type for a natural language question and generating its corresponding structured query over a KB. However, existing systems often rely on shallow probabilistic methods, which are less expressive than deep semantic representation methods. In addition, since KBs are still far from complete, it is necessary to develop a new strategy that leverages unstructured resources outside of KBs. In this work, we propose a novel Question Answering method with Relation Detection and Textual Evidence (QARDTE). First, to address the semantic gap problem in relation detection, we use bidirectional long-short term memory networks with different levels of abstraction to better capture sentence structures. Our model achieves improved results with robustness against a wide diversity of expressions and questions with multiple relations. Moreover, to help compensate for the incompleteness of KBs, we utilize external unstructured text to extract additional supporting evidence and combine this evidence with relation information during the answer re-ranking process. In experiments on two well-known benchmarks, our system achieves F1 values of 0.558 (+2.8%) and 0.663 (+5.7%), which are state-of-the-art results that show significant improvement over existing KBQA systems.

Partial Text

Question answering (QA) has long been an important research topic in natural language processing. A factoid QA system is designed to automatically answer a factoid question with concise and accurate answers about objective facts. In recent years, as large-scale knowledge bases (KBs) such as Freebase [1], YAGO [2], and DBpedia [3] have been developed, QA systems have started to use these KBs as important resources to access general knowledge in a clean and structured format. The development of a highly accurate knowledge-based question answering (KBQA) system could be beneficial in many fields, such as medical treatment, voice assistance and consumer self-service.

Typically, methods of performing a factoid QA task can be categorized into two types based on the data they use: (1) QA methods based on structured data and (2) QA methods based on unstructured data.

In this section, we introduce the experimental setup, the main results and a detailed analysis of our system.

In this study, we have proposed a KBQA method named QARDTE, which integrates our improved relation detection model and a textual evidence extractor to enhance the final results. The relation detection model is built on a Bi-LSTM network, which has been proven to demonstrate better performance in capturing sentence structures than any other commonly used network. We feed the network with different levels of abstraction to achieve better sentence representation. Moreover, we utilize external unstructured text to extract additional supporting evidence. Combining the information obtained from the relation detection results and the extracted textual evidence during the answer ranking process yields improved results. Experiments on two QA benchmarks, WebQuestions and Free917, show that our method achieves significant improvements compared with existing KBQA systems.




0 0 vote
Article Rating
Notify of
Inline Feedbacks
View all comments