Trends and challenges of multimodal user interfaces

Abstract

The development of information technologies brings new possibilities for its users. The development is noticeable in user interface design as well – new data input and output methods and facilities are proposed, increasing resource capabilities allow more realistic user interfaces. However together with the growth of user interface possibilities and technologies new challenges rise too. Therefore it is important to understand current challenges in order to understand future trends. In this paper we review the peculiarities of multimodal user interfaces and propose multimodal user interface taxonomy. This taxonomy defines possible types of data input and output and is used for input/output combination matrix generation. We summarize existing solutions and challenges based on the matrix and evaluate each combination. The evaluated combinations in the matrix highlight the main tendencies in natural user interface design area.