Evaluating an intelligent Q&A system for mobile cultural learning

Article

Doumanis, I. and Smith, S. 2015. Evaluating an intelligent Q&A system for mobile cultural learning. EAI Endorsed Transactions on Future Intelligent Educational Environments. 15 (2). https://doi.org/10.4108/fiee.1.2.e3
TypeArticle
TitleEvaluating an intelligent Q&A system for mobile cultural learning
AuthorsDoumanis, I. and Smith, S.
Abstract

The paper presents a user study designed to evaluate the impact of an intelligent Q&A system for mobile cultural learning on the user’s subjective impressions and retention performance. We have chosen a recent pedagogical framework for mobile learning as a theoretical foundation for this work and designed our system accordingly. In particular, our Q&A system enables individual learners to interact with unknown cultural content under simulated mobile conditions in a well-organised and structured manner using natural language. To investigate the impact of the Q&A system on the participants’ retention of cultural content, we compared two variations of the Q&A system in the lab under simulated mobile conditions. The systems differed both in terms of the approach for processing natural language and style of a Q&A session. Below, we present the results of the study and a series of design recommendations that should aid in the development of more robust Q&A models for mobile cultural systems.

Keywordsassisted learning, intelligent interaction, pervasive & ubiquitous education, technology supported education, culturalinformation
Research GroupResearch Group on Development of Intelligent Environments
PublisherICST
JournalEAI Endorsed Transactions on Future Intelligent Educational Environments
ISSN2409-0034
Publication dates
Print16 Jun 2015
Publication process dates
Deposited10 Jul 2015
Accepted28 Aug 2014
Output statusPublished
Publisher's version
License
Copyright Statement

Copyright © 2015 I. Doumanis and S. Smith, licensed to ICST. This is an open access article distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/3.0/), which permits unlimited use, distribution and reproduction
in any medium so long as the original work is properly cited.

Additional information

EAI Endorsed Transactions on Future Intelligent Educational Environments 15(2): e3

Digital Object Identifier (DOI)https://doi.org/10.4108/fiee.1.2.e3
LanguageEnglish
Permalink -

https://repository.mdx.ac.uk/item/85qyw

Log in to edit

Download files

Publisher's version
fiee.1.2.e3.pdf
License: CC BY 4.0

  • 233
    total views
  • 83
    total downloads
  • 4
    views this month
  • 8
    downloads this month

Export as

Related outputs

Cloistered knowledge capture and retrieval: offline LLMs and vector search for enterprise
Hall, S.J., Smith, S., Başkent, C. and de Raffaele, C. 2025. Cloistered knowledge capture and retrieval: offline LLMs and vector search for enterprise. 5th International Conference on Smart Information Systems and Technologies. Astana, Kazakhstan 14 - 16 May 2025 IEEE. https://doi.org/10.1109/sist61657.2025.11139171
Lessons from adopting a competency-based assessment approach for an introductory programming module
Androutsopoulos, K., Heeney, M., Smith, S. and Ticar, M.A. 2025. Lessons from adopting a competency-based assessment approach for an introductory programming module. ITiCSE 2025: Innovation and Technology in Computer Science Education. Nijmegen, Netherlands 27 Jun - 02 Jul 2025 Association for Computing Machinery (ACM). https://doi.org/10.1145/3724389.3731257
Evaluating the effectiveness of a collaborative framework on student learning
Zammit, O., Smith, S. and De Raffaele, C. 2024. Evaluating the effectiveness of a collaborative framework on student learning. 2024 International Conference on Machine Learning and Artificial Intelligence. Edinburgh, UK 21 - 23 Oct 2024 Pages Conferences.
Optimising student internet navigation: a comparative analysis of machine learning algorithms for action prediction
Zammit, O., Smith, S. and De Raffaele, C. 2024. Optimising student internet navigation: a comparative analysis of machine learning algorithms for action prediction. The Future of Education - 14th Edition International Conference. Florence, Italy 20 - 21 Jun 2024 Filodiritto Editore.
Adaptive PMME medium access control protocol for multi-event IoT sensor networks
Hang, N.T.T., Nguyen, H. and Smith, S. 2024. Adaptive PMME medium access control protocol for multi-event IoT sensor networks. 7th International Conference on Advanced Communication Technologies and Networking (CommNet). Rabat, Morocco 04 - 07 Dec 2024 IEEE. https://doi.org/10.1109/CommNet63022.2024.10793281
A taxonomy of knowledge management systems in the micro-enterprise
Hall, S.J., Smith, S., Baskent, C. and De Raffaele, C. 2024. A taxonomy of knowledge management systems in the micro-enterprise. International Journal of Operations Management. 4 (1), pp. 7-20. https://doi.org/10.18775/ijom.2757-0509.2020.41.4001
Knowledge management for the micro enterprise: a taxonomy
Hall, S., Smith, S., Baskent, C. and De Raffaele, C. 2023. Knowledge management for the micro enterprise: a taxonomy. Matos, F. and Rosa, A. (ed.) 24th European Conference on Knowledge Management. Lisboa, Portugal 07 - 08 Sep 2023 Academic Conferences International (ACI). https://doi.org/10.34190/eckm.24.1.1268
Comparison of the effectiveness and performance of student workgroups in online wiki activities with and without AI
Nalli, G. and Smith, S. 2023. Comparison of the effectiveness and performance of student workgroups in online wiki activities with and without AI. 4th International Electronic Conference on Applied Sciences. Online 27 Oct - 10 Nov 2023 MDPI. https://doi.org/10.3390/ASEC2023-16273
Reducing the dependency of having prior domain knowledge for effective online information retrieval
Zammit, O., Smith, S., Windridge, D. and De Raffaele, C. 2023. Reducing the dependency of having prior domain knowledge for effective online information retrieval. Expert Systems. 40 (4). https://doi.org/10.1111/exsy.13014
Artificial Intelligence to improve learning outcomes through online collaborative activities
Nalli, G., Amendola, D. and Smith, S. 2022. Artificial Intelligence to improve learning outcomes through online collaborative activities. Fotaris, P. and Blake, A. (ed.) 21st European Conference on e-Learning. Brighton, UK 27 - 28 Oct 2022 Academic Conferences International (ACI). pp. 475-479 https://doi.org/10.34190/ecel.21.1.661
Designing a marker set for vertical tangible user interfaces
Attard, G., De Raffaele, C. and Smith, S. 2021. Designing a marker set for vertical tangible user interfaces. Jaber, K. (ed.) 10th ICIT 2021: Advanced Machine Learning and Deep Learning. Amman, Jordan 14 - 15 Jul 2021 IEEE. pp. 974-979 https://doi.org/10.1109/ICIT52682.2021.9491659
Exposing students to new terminologies while collecting browsing search data (best technical paper)
Zammit, O., Smith, S., Windridge, D. and De Raffaele, C. 2020. Exposing students to new terminologies while collecting browsing search data (best technical paper). 40th SGAI International Conference on Artificial Intelligence, AI 2020. Cambridge, UK 15 - 17 Dec 2020 Cham Springer. https://doi.org/10.1007/978-3-030-63799-6_1
TangiBoard: a toolkit to reduce the implementation burden of Tangible User Interfaces in education
Attard, G., De Raffaele, C. and Smith, S. 2019. TangiBoard: a toolkit to reduce the implementation burden of Tangible User Interfaces in education. The 13th IEEE International Conference Application of Information and Communication Technologies (AICT 2019). Baku, Azerbaijan 23 - 25 Oct 2019 IEEE. pp. 1-7 https://doi.org/10.1109/AICT47866.2019.8981773
Exposing knowledge: providing a real-time view of the domain under study for students
Zammit, O., Smith, S., De Raffaele, C. and Petridis, M. 2019. Exposing knowledge: providing a real-time view of the domain under study for students. Bramer, M. and Petridis, M. (ed.) AI-2019 Thirty-ninth SGAI International Conference on Artificial Intelligence. Cambridge, United Kingdom 17 - 19 Dec 2019 Springer. pp. 122-135 https://doi.org/10.1007/978-3-030-34885-4_9
An active tangible user interface framework for teaching and learning artificial intelligence
De Raffaele, C., Smith, S. and Gemikonakli, O. 2018. An active tangible user interface framework for teaching and learning artificial intelligence. ACM IUI 2018: 23rd ACM Conference on Intelligent User Interfaces. Tokyo, Japan 07 - 11 Mar 2018 Association for Computing Machinery (ACM). pp. 535-546 https://doi.org/10.1145/3172944.3172976
Testing literacy educational software to develop design guidelines for children with Autism
Tuedor, M., Franco, F., White, A.S., Smith, S. and Adams, R. 2018. Testing literacy educational software to develop design guidelines for children with Autism. International Journal of Disability, Development and Education. 66 (1), pp. 19-35. https://doi.org/10.1080/1034912X.2018.1450494
The application of Tangible User Interfaces of teaching and learning in Higher Education
De Raffaele, C., Smith, S. and Gemikonakli, O. 2017. The application of Tangible User Interfaces of teaching and learning in Higher Education. in: Branch, J., Hayes, S., Hørsted, A. and Nygaard, C. (ed.) Innovative Teaching and Learning in Higher Education Oxfordshire, UK Libri Publishing. pp. 215-226
Enabling the effective teaching and learning of advanced robotics in higher education using an active TUI framework
De Raffaele, C., Smith, S. and Gemikonakli, O. 2017. Enabling the effective teaching and learning of advanced robotics in higher education using an active TUI framework. 3rd Africa and Middle East Conference on Software Engineering. Cairo, Egypt 12 - 13 Dec 2017 Association for Computing Machinery (ACM). pp. 7-12 https://doi.org/10.1145/3178298.3178299
Explaining multi-threaded task scheduling using tangible user interfaces in higher educational contexts
De Raffaele, C., Smith, S. and Gemikonakli, O. 2017. Explaining multi-threaded task scheduling using tangible user interfaces in higher educational contexts. 2017 IEEE Global Engineering Education Conference (EDUCON). Athens, Greece 25 - 28 Apr 2017 IEEE. pp. 1383-1390 https://doi.org/10.1109/EDUCON.2017.7943028
Bridging the digital divide for e-learning students through adaptive VLEs
Spiteri, C., De Raffaele, C. and Smith, S. 2016. Bridging the digital divide for e-learning students through adaptive VLEs. 2016 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE 2016). Bangkok, Thailand 07 - 09 Dec 2016 IEEE. pp. 210-217 https://doi.org/10.1109/TALE.2016.7851796
Teaching and learning queueing theory concepts using Tangible User Interfaces
De Raffaele, C., Smith, S. and Gemikonakli, O. 2016. Teaching and learning queueing theory concepts using Tangible User Interfaces. 2016 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE 2016). Bangkok, Thailand 07 - 09 Dec 2016 IEEE. pp. 194-201 https://doi.org/10.1109/TALE.2016.7851794
The aptness of tangible user interfaces for explaining abstract computer network principles
De Raffaele, C., Smith, S. and Gemikonakli, O. 2016. The aptness of tangible user interfaces for explaining abstract computer network principles. FIE 2016 -The 46th Annual Frontiers in Education (FIE) Conference. Erie, PA, USA 12 - 15 Oct 2016 IEEE. https://doi.org/10.1109/FIE.2016.7757573
An expert review of REVERIE and its potential for game-based learning
Doumanis, I., Porter, S., Economou, D. and Smith, S. 2015. An expert review of REVERIE and its potential for game-based learning. Preuveneers, D. (ed.) The 11th International Conference on Intelligent Environments - IE'15. Prague 13 - 14 Jul 2015 IOS Press. pp. 306-313 https://doi.org/10.3233/978-1-61499-530-2-306
The use of social networking sites in eLearning
De Raffaele, C., Bugeja, L. and Smith, S. 2015. The use of social networking sites in eLearning. International Journal of Social, Behavioral, Educational, Economic and Management Engineering. 9 (4), pp. 1223-1228.
Assessment techniques in cross-cultural usability evaluation
Jagne, J., Smith, S. and Duncker-Gassen, E. 2006. Assessment techniques in cross-cultural usability evaluation. IADIS WWW/Internet International Conference 2006. Murcia, Spain
Intelligent environments simulations, towards a smart campus
Campuzano, F., Doumanis, I., Smith, S. and Botia, J. 2014. Intelligent environments simulations, towards a smart campus. The University as a Context Platform, Workshop. University of West London, London, United Kingdom 14 Nov 2014 https://doi.org/10.13140/2.1.3141.3447
Evaluating the impact of embodied conversational agents (ECAs) attentional behaviors on user retention of cultural content in a simulated mobile environment
Doumanis, I. and Smith, S. 2014. Evaluating the impact of embodied conversational agents (ECAs) attentional behaviors on user retention of cultural content in a simulated mobile environment. GazeIn '14: 7th Workshop on Eye Gaze in Intelligent Human Machine Interaction: Eye-Gaze & Multimodality. Istanbul, Turkey 16 - 16 Nov 2014 Association for Computing Machinery (ACM). pp. 27-32 https://doi.org/10.1145/2666642.2666650
Evaluating the impact of ECAs on user navigation performance and perception
Doumanis, I. and Smith, S. 2013. Evaluating the impact of ECAs on user navigation performance and perception. 13th International Conference on Intelligent Virtual Agents. Edinburgh, United Kingdom 29 - 31 Aug 2013 Springer. pp. 474-475
Beyond Artificial Intelligence Markup Language (AIML) rapid prototyping of a Q&A system
Doumanis, I. and Smith, S. 2013. Beyond Artificial Intelligence Markup Language (AIML) rapid prototyping of a Q&A system. 1st International Workshop on Museums as Intelligent Environments (MASIE'13). Athens, Greece 16 - 17 Jul 2013 IOS Press. pp. 530-540 https://doi.org/10.3233/978-1-61499-286-8-530
An empirical study on the effects of embodied conversational agents on user retention performance and perception in a simulated mobile environment
Doumanis, I. and Smith, S. 2013. An empirical study on the effects of embodied conversational agents on user retention performance and perception in a simulated mobile environment. Botía, J. and Charitos, D. (ed.) 2nd Workshop on Future Intelligent Educational Environments Conference (WOFIEE’13). Athens, Greece 16 - 17 Jul 2013 IOS Press. pp. 431-442 https://doi.org/10.3233/978-1-61499-286-8-431
Embedding information literacy skills as employability attributes
Smith, S. and Edwards, J. 2012. Embedding information literacy skills as employability attributes. LILAC 2012. Glasgow, Scotland 11 - 13 Apr 2012
Embedded writing instruction in the first year curriculum
Bernaschina, P. and Smith, S. 2012. Embedded writing instruction in the first year curriculum. Journal of Learning Development in Higher Education. https://doi.org/10.47408/jldhe.v0i0.186
Embedding information literacy skills as employability attributes
Smith, S. and Edwards, J. 2012. Embedding information literacy skills as employability attributes. ALISS Quarterly. 7 (4), pp. 22-27.
Human-computer interaction: users, tasks and designs
Adams, R. and Smith, S. 2005. Human-computer interaction: users, tasks and designs. London Middlesex University Press.
Paradigms, technology and demand characteristics for universal access: can we treat different technologies the same?
Adams, R. and Smith, S. 2005. Paradigms, technology and demand characteristics for universal access: can we treat different technologies the same? in: HCI international 2005 11th international conference on human-computer interaction: proceedings. Mahwah, NJ. Lawrence Erlbaum Associates..
Cross-cultural system design strategy: avoiding problems in usability evaluation
Jagne, J., Duncker-Gassen, E. and Smith, S. 2006. Cross-cultural system design strategy: avoiding problems in usability evaluation. IADIS WWW/Internet International Conference 2006. Murcia, Spain pp. 88-92