• MARINO LINAJE Quercus Software Engineering Group. School of Technology. University of Extremadura, Spain
  • JUAN CARLOS PRECIADO Quercus Software Engineering Group. School of Technology. University of Extremadura, Spain
  • ROBERTO RODRIGUEZ-ECHEVERRIA Quercus Software Engineering Group. School of Technology. University of Extremadura, Spain
  • JOSÉ MARÍA CONEJERO Quercus Software Engineering Group. School of Technology. University of Extremadura, Spain
  • FERNANDO SÁNCHEZ-FIGUEROA Quercus Software Engineering Group. School of Technology. University of Extremadura, Spain


Design tools and techniques, Web Engineering, Rich Internet Applications, Multimedia temporal relationships, SMIL Timesheets


Temporal behaviors are being incorporated into the user interfaces of Web applications making them look more and more like multimedia applications, the so-called Rich Internet Application (RIA) user interfaces. Due to RIA complexity, some research communities have proposed models to ease its development. However, there is a gap to cover between formal temporal relationships and the current state of the art in the RIA model-driven development techniques. The purpose of this paper is to specify a temporal behavioral model for data-intensive RIA user interfaces with three main objectives. The first one is that the model must be usable by non-experts in engineering specifications (e.g., Web designers). The second one is that the model must be suitable to be implemented in a CASE tool integrating temporal behaviors in the RIA model driven development workflow. The third one is that the temporal behaviors specified must run in current Web browsers. The approach here presented is based on SMIL Timesheets, a standard that can be used as a foundation to extend RIA user interface model driven proposals.


Download data is not yet available.


Abdelli, A. and Badache, N., Context-aware adaptation of multimedia documents for consistent

presentations. Multimedia systems 17(6), (2011), 465–486

Adobe: Flash Builder. Available at:

Allen, J., Maintaining knowledge about temporal intervals. Communications of the ACM 26(11),

(1983), 832–843

Antonacci, M., Muchaluat-Saade, D., Rodrigues and R., Soares, L., Improving the expressiveness of

XML-based hypermedia authoring languages. In: Multimedia Modeling Conference. Nagano,

Japan (2000). DOI 10.1142/9789812791993 0006

Aubert, O., Champin, P., Prié, Y. and Richard, B., Canonical processes in active reading and

hypervideo production. Multimedia Systems 14(6), (2008), 427–433

Bertolotti, P. and Gaggi, O., A study on multimedia documents behavior: a notion of equivalence.

Multimedia Tools and Applications 33, (2007), 301–324, DOI 10.1007/s11042007-0102-2

Bozzon A. Comai S., Fraternali P. and Toffetti G., Conceptual Modeling and Code Generation for

Rich Internet Applications. In: International Conference on Web Engineering (ICWE), ACM 263,

Palo Alto, USA, (2006), 353-360.

Bulterman, D., SMIL 2.0 part 1: overview, concepts, and structure. IEEE Multimedia 8, (2002), 82–

, DOI 10.1109/93.959106

Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L. and Vanderdonckt, J., A unifying

reference framework for multi-target user interfaces. Interacting with Computers 15(3), (2003),


Cazenave, F., Quint, V. and Roisin, C., Timesheets.js: when SMIL meets HTML5 and CSS3. In:

Proceedings of the 11th ACM symposium on Document engineering, DocEng ’11, ACM, NY,

USA, (2011), 43–52, DOI 10.1145/2034691.2034700

Chung, S. M., and Pereira, A. L., Timed Petri net representation of SMIL. IEEE MultiMedia,

(1), (2005), 64-72

Deltour, R. and Roisin, C., The limsee3 multimedia authoring model. In: Proceedings of the 2006

ACM symposium on Document engineering, DocEng ’06, ACM, NY, USA, (2006), pp. 173–175,

DOI 10.1145/1166160.1166203

Fraternali P., Comai S., Bozzon A. and Toffetti G., Engineering rich internet applications with a

model-driven approach. ACM Trans. Web 4, 2, Article 7, (2010)

Gaggi, O. and Bossi, A., Analysis and verification of SMIL documents. Multimedia systems 17(6),

(2011), 487–506

Helms, J. and Abrams, M., Retrospective on UI description languages, based on eight years

experience with the User Interface Markup Language UIML. Int. J. Web Eng. Technol. 4, (2008),

–162, DOI 10.1504/IJWET.2008.018095

Hendrickx, F., Beckers, T., Oorts, N. and de Walle, R.V., An integrated approach for device

independent publication of complex multimedia documents. In: Internet and Multimedia Systems

and Applications, (2005), 347–352

Jansen, J. and Bulterman, D.C., SMIL State: an architecture and implementation for adaptive timebased

web applications. Multimedia Tools and Applications 43, (2009), 203–224, DOI


Koch N., Pigerl M., Zhang G. and Morozova T., Patterns for the Model-Based Development of

RIAs. In: International Conference on Web Engineering (ICWE), Springer LNCS 5648, San

Sebastián, Spain, (2009), 283-291.

Kuijk, F., Guimara˜es, R.L., Cesar, P. and Bulterman, D.C., Adding dynamic visual manipulations

to declarative multimedia documents. In: 9th ACM Symposium on Document engineering,

DocEng ’09, ACM, NY, USA, (2009), 149–152, DOI 10.1145/1600193.1600227

LaLomia, M. and Sidowski, J., Measurements of computer satisfaction, literacy, and aptitudes: A

review. International Journal of Human-Computer Interaction 2(3), (1990), 231–253

Lalomia, M. and Spitzberg, B., Preliminary Development of a Model and Measure of Computer-

Mediated Communication (CMC) Competence. Journal of Computer-Mediated Communication,

, (2006), 629–666, doi:10.1111/j.1083-6101.2006.00030.

Linaje M., Sanchez-Figueroa, F. and Preciado, J., RUX-Method temporal schema specification.

Available at:

Linaje, M., Preciado, J.C., Morales-Chaparro, R., Rodriguez-Echeverria, R. and Sanchez-Figueroa,

F., Automatic Generation of RIAs Using RUX-Tool and Webratio. In: 9th International

Conference on Web Engineering, ICWE ’9, Springer-Verlag, Berlin, Heidelberg, (2009), 501–504,

DOI 10.1007/978-3-642-02818-2 48

Linaje, M., Preciado, J.C. and Sanchez-Figueroa, F., Domain-specific model for designing rich

internet application user interfaces. In: V. Lopez Jaquero, F. Montero Simarro, J.P. Molina Masso,

J. Vanderdonckt (eds.) Computer-Aided Design of User Interfaces VI, Springer London, (2009),

–306, DOI 10.1007/978-1-84882-206-1 27

Linaje, M., Preciado, J.C. and Sanchez-Figueroa, F., Engineering rich internet application user

interfaces over legacy web models. IEEE Internet Computing 11, (2007), 53–59 DOI


Linaje, M. and Preciado, J.C., Multi-device context-aware RIAs using a model-driven approach.

Journal of Universal Computer Science 16(15), (2010), 2038–2059

Linaje, M., Rux-method: modelado de interfaces de usuario web multidispositivo, multimedia,

interactivas y accesibles. Ph.D. thesis, Universidad de Extremadura, (2009)

Ma, H., Shin, K., Checking consistency in multimedia synchronization constraints. IEEE

Transactions on Multimedia, 6(4), (2004), 565–574

Meliá S., Gómez J., Pérez S. and Díaz O., A Model-Driven Development for GWT-Based Rich

Internet Applications with OOH4RIA. In: International Conference on Web Engineering (ICWE),

New York, USA, (2008), 13-23.

Meliá S., Gómez J., Pérez S. and Díaz O., Architectural and Technological Variability in Rich

Internet Applications. IEEE Internet Computing 14, 3 (2010), 24-32.

Microsoft: Expression Blend. Available at:

Mikác, J., Roisin, C. and Le Duc, B., An export architecture for a multimedia authoring

environment. In: Proceeding of the eighth ACM symposium on Document engineering, DocEng

’08, ACM, NY, USA (2008), 28–31 DOI 10.1145/1410140.1410147

Montero, S., Fernandez, C., Dodero, J., Aedo, I. and Diaz, P., A design toolkit for hypermedia

applications based on Ariadne development method. In: R. Jacob, Q. Limbourg, J. Vanderdonckt

(eds.) Computer-Aided Design of User Interfaces IV, Springer Netherlands, (2005), 43–54, DOI

1007/1-4020-3304-4 4

Nielsen, J. and Hackos, J., Usability engineering, vol. 125184069. Academic press San Diego,


Nouguier, E., Timesheets Scheduler. Available at:


Pellan, B. and Concolato, C., Authoring of scalable multimedia documents. MultimediaTools and

Applications 43, (2009), 225–252 DOI 10.1007/s11042-009-0268-x

Pihkala, K. and Vuorimaa, P., Nine methods to extend SMIL for multimedia applications.

Multimedia Tools and Applications 28, 51/67, (2006), DOI 10.1007/s11042-006-5120-y

Pimentel, M.d.G., Cattelan, R.G., Melo, E.L., Prado, A.F. and Teixeira, C.A.C., Ubiquitous enduser

live editing of interactive multimedia programs. In: 14th Brazilian Symposium on Multimedia

and the Web, WebMedia ’08, ACM, NY, USA, (2008), 123–129, DOI 10.1145/1666091.1666113

Pleuß, A., Botterweck, G. and Hußmann, H., Modeling advanced concepts of interactive

multimedia applications. In: Visual Languages and Human-Centric Computing, (2009), 31–38

DOI 10.1109/VLHCC.2009.5295305

Pleuß, A. and Hußmann, H., Integrating authoring tools into model-driven development of

interactive multimedia applications. In: 12th Int. Conf. on Human-computer interaction: interaction

design and usability, HCI’07, Springer-Verlag, Berlin, Heidelberg, (2007), 1168–1177

Pleuß, A., MML: a language for modeling interactive multimedia applications. In: 7th IEEE

International Symposium on Multimedia, (2005), 9–17 DOI 10.1109/ISM.2005.80

Preciado, J.C., Linaje, M., Sanchez, F. and Comai, S., Necessity of methodologies to model rich

internet applications. In: 7th IEEE International Symposium on Web Site Evolution, IEEE

Computer Society, Washington, DC, USA, (2005), 7–13, DOI 10.1109/WSE.2005.10

Rodrigues, L., Antonacci, M., Rodrigues, R., Muchaluat-Saade, D. and Soares, L., Improving

SMIL with NCM Facilities. Multimedia Tools and Applications 16, (2002), 29–54 DOI


Santanchè, A., Medeiros, C. and Pastorello, G., User-author centered multimedia building blocks.

Multimedia systems 12(4), (2007), 403–421

Sauer, S. and Engels, G., Extending UML for modeling of multimedia applications. In: IEEE

Symposium on Visual Languages, IEEE Computer Society, Washington, DC, USA, (1999), 80–


Scherp, A., Canonical processes for creating personalized semantically rich multimedia

presentations. Multimedia Systems 14(6), (2008), 415–425

Schwinger, W., Retschitzegger, W., Schauerhuber, A., Kappel, G., Wimmer, M., Pr¨oll,B., Castro,

C.C., Casteleyn, S., Troyer, O.D., Fraternali, P., et al., A survey on web modeling approaches for

ubiquitous web applications. International Journal of Web Information Systems 4(3), (2008), 234–

Silva, H.V.O., Rodrigues, R.F., Soares, L.F.G. and Muchaluat Saade, D.C., NCL 2.0: integrating

new concepts to XML modular languages. In: Proceedings of the 2004 ACM symposium on

Document engineering, ACM, NY, USA (2004), 188–197, DOI 10.1145/1030397.1030433

Specht, G. and Zoller, P., HMT: Modeling Temporal Aspects in Hypermedia Applications. In: 1st

Int. Conf. on Web-Age Information Management, WAIM ’00, Springer-Verlag, London, UK,

(2000), 259–270

Sung, M. and Lee, D., A collaborative multimedia authoring system. In: M. Li, X.H. Sun,Q. Deng,

J. Ni (eds.) Grid and Cooperative Computing, Lecture Notes in Computer Science, vol. 3033,

Springer Berlin / Heidelberg, (2004), 311–318 DOI 10.1007/9783-540-24680-0 54

Thompson, S., King, P. and Cameron, H., Modelling reactive multimedia: Design and authoring.

Multimedia Tools and Applications 27, (2005), 23–52 DOI 10.1007/s11042005-2713-9

Toffetti, G., Comai, S., Preciado, J.C. and Trigueros, M.L., State-of-the-art and trends in the

systematic development of rich internet applications. Journal of Web Engineering 10(1), (2011),


Vuorimaa, P., Timesheets JavaScript Engine. Available

W3C: SMIL 3 (2008). Available at:

W3C: SMIL Timesheets 1.0 (2012). Available at:

W3C: Synchronized Multimedia Authoring Tools. Available at:

Willrich, R., De Saqui-Sannes, P., Sénac, P. and Diaz, M., Multimedia authoring with hierarchical

timed stream Petri nets and java. Multimedia Tools and Applications 16, (2002), 7–27 DOI


Yang, C. C., Wang, Y. C. and Tien, C. W., Synchronization modeling and its application for

SMIL2. 0 presentations. Journal of Systems and Software, 80(7), (2007), 1142-1155

Yang, C.C., Chu, C.K. and WANG, Y.C., Extension of Timeline-based Editing for

Nondeterministic Temporal Behavior in SMIL2.0 Authoring. Journal of information science and

engineering 24(5), (2008), 1377–1395

Yang, C.C. and Yang, Y.Z., SMILAuthor: an authoring system for SMIL-based multimedia

presentations. Multimedia Tools and Applications 21, (2003), 243–260 DOI