research project: Approach for Genericity in Assistance To complEx tasks


User assistance, language, model, generic approach, epiphytic approach, knowledge engineering

Academic members

Stéphanie Jean-Daubias, Pierre-Antoine Champin, Blandine Ginon, Marie Lefevre
Academic members Aidaxis


Brice Buffa (M1 internship, 2013), Léa Bonneville (3rd year of INSA internship, 2014), Raphaël Diana (M1&M1 internship, 2015 & 2016), Loïc and Thibaut Salanon (M1 internship, 2015), Yoni Sebban (M1 internship, 2015), Blandine Ginon (PhD thesis, 2011-2014), Le Vinh Thai (PhD thesis, 2013-2017)
News Through a collaboration with LIRIS-TWEAK, the Aidaxis company launched its software solution for user assistance

The AGATE project

SEPIA process

The AGATE project (an Approach for Genericity in Assistance To complEx tasks) aims to propose generic models and unified tools to make possible the setup of assistance systems in any existing application, that we call the target-application.

For this purpose, we proposed an adjunction process of epi-assistance system in a target-application in two phases. The first phase involves an assistance designer: an expert of the target-application who wishes to design an assistance system. This phase enables the assistance designer to specify the assistance that he/she wishes for the target-application, by defining a set of assistance rules. The second phase involves the target-application end-users. It consists in the execution of the assistance specified by the designer. This phase occurs at each use of the target-application by an end-user; it is composed of three processes. The monitoring of the target-application exploits a set of epi-detectors that enable the continuous observation and the tracing of all interactions between the user and the target-application interface. In parallel, the process identifying assistance needs exploits the assistance rules defined by the designer and triggers the process elaborating an answer to the identified assistance need. The answer is executed as one or several assistance actions, performed by an epi-assistant in the target-application.

The aLDEAS language

aLDEAS components

We propose aLDEAS (a Language to Define Epi-Assistance Systems), a graphical rule language aimed at assistance designers for defining assistance systems. We choose to propose a graphical language, very suitable to the representation by a set of simple rules of assistance systems, in particular epiphytic assistance systems.

An aLDEAS block is a labelled direct graph, with nodes and edges. It must have exactly one source, which must be a "block start", and one or several sinks, which must all be "block end". It is executed by walking the graph. aLDEAS components can be combined to create assistance actions addressing assistance needs, that will then be executed by our system SEPIA on top of the target-application.

aLDEAS is completed by several patterns that facilitate the definition of aLDEAS blocs: a rule pattern and 3 complex assistance actions (step-to-step, guided presentation and animated agent action).

The SEPIA system

Architecture of SEPIA

We implemented aLDEAS and the patterns that complete it in the SEPIA environment (Specification and Execution of Personalized Intelligent Assistance). demonstration with SEPIA-web

The assistance editor is a tool aimed at assistance designers. It implements aLDEAS and makes possible the specification, for any given target-application, of an assistance system described by a set of assistance rules complying with the rule pattern. The assistance editor provides an interface for the definition of elementary assistance action and for the definition of action that instantiates the pattern proposed. No knowledge on aLDEAS is required to use the assistance editor.
We developed a set of epi-assistants, able to perform in a target-application the elementary assistance actions proposed by aLDEAS and defined thanks to our assistance editor. The figure presents the elementary assistance actions that can be performed by our epi-assistants depending on the target-application type. The automated actions are available on any web application, on Windows native and Java applications, and on Linux GKT and Qt applications. Our epi-assistants performing these actions use a user script for web applications and different accessibility libraries for desktop applications: UIAutomation, JavaAccessibility and ATK-SPI. Enhancement and masking actions are available for the same categories of applications as they need the same technology to find the bounding rectangle of the component that must be enhanced or masked. Animated agent actions are available on any Windows, Web or desktop application. Messages and resources launch are available for any application.

Research interests

  • What is the best moment to propose assistance? (with Simone Stumpf, City University London)

  • Change management and multicultural  assistance for a course outline device (PEPS ONDES project, with Catherine Loisy, Pierre Bénech)

  • Assistance for people with disabilities

Examples of assistance systems setup with SEPIA

logo Aidaxis

SEPIA assistance by Aidaxis

Discover of NetBeans by students: creation of a Java graphical application

Target-application: NetBeans IDE
Context: NetBeans is used in the context of an HCM course of bachelor's dregree, at the University of Lyon 1
Associated experimentation: definition of a tutorial in 5 parts by teachers and use of the tutorial by 52 students, with a control group of 47 students
Relative publication: "Adding epiphytic assistance systems in learning applications using the SEPIA system", in EC-TEL 2014

Fixing red eyes on a photo with PhotoScape

Target-application: PhotoScape photo edition freeware
Context: The aim of this assistance system was to guide end-users through the fixing of red eyes on a given photo
Associated experimentation: we ask 200 end-users to fix red eyes on a given photo with PhotoScape, without any assistance for 100 end-users of group A and with SEPIA assistance for 100 end-users of group B. Group A in divided in 2 sub-goups: group A1 for the 49 end-users that succeeded in performing the task without assistance, and group A2 for the 51 end-users that given up without assistance in a first time and that then succeeded with SEPIA assistance in a second time
Relative publication: "Setup of epiphytic assistance systems with SEPIA", in EKAW 2014

Pedagogical assistance for dyslexic learners in Mathématiques faciles

Target-application: Mathématiques faciles web site
Context: Mathématiques faciles provides learners with calculation lessons and exercices. Its interface is overloaded and do not respect some normes intended for dyslexic learners. This assistance system simplify the interface and adapts it for dyslexics learners, by changing the colors and font and the presentation of the calculation for instance.
Contributor: Margot Fargier

Pedagogical assistance for dyslexic learners in Ortholud

Target-application: Ortholud web site
Context: Ortholud provides dyslexic learners with french spelling exercices. This assistance system guides the leaners and provided them pedagogical advices and clues
Contributor: Margot Fargier

Use of Tomuss functionnalities by different types of users

Target-application: Tomuss
Context: Tomuss is the application of notes management used at the University of Lyon 1. It can be used by teachers, for instance to input notes or to send information to students filtered about a given criteri, and it can be used by secretaries, for instance to print the students' notes or to register them to an option.
Contributor: Elodie Hoarau

Handling Fovea MOOC

Target-application: Fovea MOOC
Context: COAT project
Contributor: Léa Bonneville

Assistance to teachers in the implementation of a program approach

Target-application: @LOES + web sites
Context: CRAIES project - UNR-RA
Contributor: Margot Fargier - Amandine Raze - Pierre Bénech

Setup of assistance in various pedagogical software

Target-applications: pedagogical software designed by students in
M1if22 ILE course
Contributors: Students from ILE course in master's degree, at University of Lyon
Associated experimentation:

More examples...


  • Un tableau de bord pour le concepteur de systèmes d’assistance, R. Diana, B. Ginon & S. Jean-Daubias, Quels tableaux de bord pour les acteurs de l’éducation ?, 6 juin 2017, Strasbourg (France). pdf
  • Langage et outils pour la spécification et l’exécution d’assistance à l’utilisateur dans des applications existantes B. Ginon, S. Jean-Daubias, P.A. Champin & M. Lefevre, Revue des Sciences et Technologies de l'Information - Série RIA : Revue d'Intelligence Artificielle, vol. 30, n°6, pp. 705-733. pdf
  • Towards the Right Assistance at the Right Time for Using Complex Interfaces, B. Ginon, S. Stumpf & S. Jean-Daubias (2016) International working conference on Advanced Visual Interfaces, 10 juin 2016, Bari (Italie), pp. 240-243. pdf
  • Modèles et outils génériques pour mettre en place des systèmes d’assistance épiphytes.   B. Ginon. Thèse de doctorat de l'Université de Lyon, 2014. pdf
  • Mise en place d’un système d’assistance personnalisée dans une application existante. B. Ginon, S. Jean-Daubias, P-A. Champin. Dans Ingénierie des connaissances, Lille, France. 2013. pdf

  • Adjonction de systèmes d’assistance personnalisée à des EIAH existants. B. Ginon, S. Jean-Daubias, P-A. Champin. Dans EIAH, Toulouse, France. pp. 15-16. 2013. pdf

  • Towards a generic model for user assistance. B. Ginon. In Doctoral Consortium of User Modeling, Adaptation, and Personalization, Montréal, Canada. pp. 356-360, 2012. pdf

  • Une assistance générique pour les utilisateurs d’applications fortement orientées connaissances. B. Ginon, S. Jean-Daubias, P-A. Champin. Dans Ingénierie des Connaissances, Paris, France. pp. 167-174. ISBN 978-2-7466-4577-6. 2012. pdf

SEPIA system
  • Setup of epiphytic assistance systems with SEPIA. B. Ginon, S. Jean-Daubias, P.-A. Champin, M. Lefevre, In conference EKAW, demonstration, Linköping, Suède, novembre 2014. pdf

  • Collecting fine-grained use traces in any application without modifying it. B. Ginon, P-A. Champin, S. Jean-Daubias. In workshop EXPPORT from the conference ICCBR, New York, USA. 2013. pdf

aLDEAS language
  • aLDEAS : a language to define epiphytic assistance systems. B. Ginon, S. Jean-Daubias, P.-A. Champin, M. Lefevre, In conference EKAW, Linköping, Suède, novembre 2014. pdf

  • aLDEAS : un langage de définition de systèmes d’assistance épiphytes. B. Ginon, S. Jean-Daubias, P-A. Champin, M. Lefevre. Dans Ingénierie des Connaissances, Clermont Ferrand. pp. 137-148. 2014 (best paper). pdf

use in educative context
  • Modèles et outils pour favoriser l’articulation entre la généricité d’un assistant intelligent et les spécificités de son usage en EIAH, L. V. Thai, thèse de l'Université de Lyon (soutenue le 23 juin 2017). pdf
  • Model of articulation between elements of a pedagogical assistance,  L. V. Thai, S. Jean-Daubias, B. Ginon & M. Lefevre, 11th European Conference on Technology Enhanced Learning (EC-TEL 2016), 16 septembre 2016, Lyon (France), pp. 656-659. pdf
  • Model of Articulation between aLDEAS Assistance Rules, L. V. Thai, S. Jean-Daubias, M. Lefevre & B. Ginon, DC CSEDU 2016 - Doctorial Consortium of the 8th International Conference on Computer Supported Education, 23 avril 2016, Rome (Italie). pdf
  • Concevoir un guidage pédagogique à travers un système d'assistance épiphyte, L. V. Thai, RJC EIAH, 17 juin 2016, Montpellier (France), pp. 79-80. pdf
  • Modèle d'articulation entre les règles définissant un système d'assistance aLDEAS,  L. V. Thai, B. Ginon, S. Jean-Daubias, M. Lefevre & P.-A. Champin (2015). « RJCIA - 13èmes Rencontres des Jeunes Chercheurs en Intelligence Artificielle, 30 juin 2015, Rennes (France), pp. 61-66. pdf
  • Adapting the SEPIA system to the educational context. L. V. Thai, B. Ginon, S. Jean-Daubias, M. Lefevre, In conference EKAW, poster, Linköping, Suède, novembre 2014. pdf

  • Adding epiphytic assistance systems in learning applications using the SEPIA system. B. Ginon, L.V. Thai, S. Jean-Daubias, M. Lefevre, P-A. Champin. In EC-TEL, Graz. 2014. pdf

  • Exploitation d’assistances épiphytes en contexte éducatif. L.V. Thai, B. Ginon. Dans RJC EIAH, La Rochelle. 2014. pdf

  • Adjonction de systèmes d’assistance personnalisée à des EIAH existantsB. Ginon, S. Jean-Daubias, P-A. Champin, Dans conférence EIAH (Environnements Informatiques pour l'Apprentissage Humain), poster, Toulouse, France, pp. 15-16, mai 2013. pdf

  • Une typologie de l'assistance aux utilisateurs : exemple d'application aux EIAH. B. Ginon, S. Jean-Daubias, P-A. Champin. Rapport de recherche RR-LIRIS-2013-007, Soumis à EIAH 2013. pdf