The architecture includes five modules, which are all developed among several project partners, and the idea consist of an unidirectional flow of data through the different modules:

Figure: To allow more flexibility, any module can connect to any other.

1. The first module is responsible for the capture of the movements coming from the user. This module is able to adapt the several capture devices: Kinect, Art-track, and Xsens MVN (Moven). The output of this module is a flow of skeleton ready to be analyzed.

2. The second module gets the data from the capture module and do an analysis to extract features, such as name, speed, amplitude, tempo, fluidity, acceleration of movements, which then form the module’s output.

3. The third module receives the features and finds the right decision, given the information/features. This Decision module is based on learning algorithms constructed using the INGREDIBLE database, and presents virtual agent’s artificial intelligence.

4. The forth module collects the decision made by the virtual agent (the Decision module) and performs the synthesis of the corresponding decided movement. The output of this module is a flow of position of a skeleton.

5. The fifth and last module gets the flow of positions and do the animation and rendering of the virtual character with which the user interacts with.