LAMDA: Large Language Model as Decision Analyst

Influence diagrams address the challenges of decision making under risk by structuring information, decisions, and values, while clearly depicting uncertainties and probabilistic dependencies. However, constructing an influence diagram requires expertise in decision analysis and is further complicated by the need to process large amounts of contextual information. This work focuses on the construction of influence diagrams from natural language input by leveraging large language models (LLMs). We design a workflow that prompts LLMs to output elements of an influence diagram and resolves issues through verification and regeneration. We also construct a new data set of typical decision problems under risk. Evaluations using this data set demonstrate that our framework effectively identifies key factors and relationships in natural language, making better decisions than standalone LLMs and LLMs enhanced with standard techniques such as chain-of-thought. Finally, we apply the large LAnguage Model as Decision Analyst (LAMDA) to discussions by groups of disease control experts on a hypothetical pandemic to demonstrate its real-world applicability. Overall, the method effectively synthesizes unstructured text into an influence diagram that, although subject to human review and refinement, enhances information processing and supports decision making.

Publisher page