Numerical reasoning is an important task in the analysis of financial documents. It helps in understanding and performing
numerical predictions with logical conclusions for the given
query seeking answers from financial texts. Recently, Large
Language Models (LLMs) have shown promising results in
multiple Question-Answering (Q-A) systems with capability
of logical reasoning. As documents related to finance often
consist of long and complex financial contexts, LLMs seem
to be best suitable for high-quality automated financial Q-A
systems. However, LLMs often face challenges in accurately
processing the various numbers within financial reports. To
be precise, extracting numerical data from the unstructured
text and semi-structured tables, and concurrently performing
accurate calculations, remains a significant bottleneck in the
financial reasoning with most of the state-of-the-art LLMs.
Recent studies have shown that structured data augmentations, such as Knowledge Graphs (KGs) have notably improved the predictions of LLMs along with logical explanations. Thus, it is an important requirement to consider inherent structured information in financial reports while using
LLMs for various financial analytics.
This paper proposes a framework to incorporate structured information using KGs along with the LLMs predictions for numerical reasoning tasks. The KGs are extracted using a proposed schema inherently from the document under processing. We evaluated our proposed framework over the benchmark data FinQA exploiting an open source LLM namely,
Llama 3.1 8B. We observed that the proposed framework improved the execution accuracy by 12% approximately.