AN SMIL-TIMESHEETS BASED TEMPORAL BEHAVIOR MODEL FOR THE VISUAL DEVELOPMENT OF WEB USER INTERFACES
Keywords:
Design tools and techniques, Web Engineering, Rich Internet Applications, Multimedia temporal relationships, SMIL TimesheetsAbstract
Temporal behaviors are being incorporated into the user interfaces of Web applications making them look more and more like multimedia applications, the so-called Rich Internet Application (RIA) user interfaces. Due to RIA complexity, some research communities have proposed models to ease its development. However, there is a gap to cover between formal temporal relationships and the current state of the art in the RIA model-driven development techniques. The purpose of this paper is to specify a temporal behavioral model for data-intensive RIA user interfaces with three main objectives. The first one is that the model must be usable by non-experts in engineering specifications (e.g., Web designers). The second one is that the model must be suitable to be implemented in a CASE tool integrating temporal behaviors in the RIA model driven development workflow. The third one is that the temporal behaviors specified must run in current Web browsers. The approach here presented is based on SMIL Timesheets, a standard that can be used as a foundation to extend RIA user interface model driven proposals.
Downloads
References
Abdelli, A. and Badache, N., Context-aware adaptation of multimedia documents for consistent
presentations. Multimedia systems 17(6), (2011), 465–486
Adobe: Flash Builder. Available at: http://www.adobe.com/devnet/flash-builder.
Allen, J., Maintaining knowledge about temporal intervals. Communications of the ACM 26(11),
(1983), 832–843
Antonacci, M., Muchaluat-Saade, D., Rodrigues and R., Soares, L., Improving the expressiveness of
XML-based hypermedia authoring languages. In: Multimedia Modeling Conference. Nagano,
Japan (2000). DOI 10.1142/9789812791993 0006
Aubert, O., Champin, P., Prié, Y. and Richard, B., Canonical processes in active reading and
hypervideo production. Multimedia Systems 14(6), (2008), 427–433
Bertolotti, P. and Gaggi, O., A study on multimedia documents behavior: a notion of equivalence.
Multimedia Tools and Applications 33, (2007), 301–324, DOI 10.1007/s11042007-0102-2
Bozzon A. Comai S., Fraternali P. and Toffetti G., Conceptual Modeling and Code Generation for
Rich Internet Applications. In: International Conference on Web Engineering (ICWE), ACM 263,
Palo Alto, USA, (2006), 353-360.
Bulterman, D., SMIL 2.0 part 1: overview, concepts, and structure. IEEE Multimedia 8, (2002), 82–
, DOI 10.1109/93.959106
Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L. and Vanderdonckt, J., A unifying
reference framework for multi-target user interfaces. Interacting with Computers 15(3), (2003),
–308
Cazenave, F., Quint, V. and Roisin, C., Timesheets.js: when SMIL meets HTML5 and CSS3. In:
Proceedings of the 11th ACM symposium on Document engineering, DocEng ’11, ACM, NY,
USA, (2011), 43–52, DOI 10.1145/2034691.2034700
Chung, S. M., and Pereira, A. L., Timed Petri net representation of SMIL. IEEE MultiMedia,
(1), (2005), 64-72
Deltour, R. and Roisin, C., The limsee3 multimedia authoring model. In: Proceedings of the 2006
ACM symposium on Document engineering, DocEng ’06, ACM, NY, USA, (2006), pp. 173–175,
DOI 10.1145/1166160.1166203
Fraternali P., Comai S., Bozzon A. and Toffetti G., Engineering rich internet applications with a
model-driven approach. ACM Trans. Web 4, 2, Article 7, (2010)
Gaggi, O. and Bossi, A., Analysis and verification of SMIL documents. Multimedia systems 17(6),
(2011), 487–506
Helms, J. and Abrams, M., Retrospective on UI description languages, based on eight years
experience with the User Interface Markup Language UIML. Int. J. Web Eng. Technol. 4, (2008),
–162, DOI 10.1504/IJWET.2008.018095
Hendrickx, F., Beckers, T., Oorts, N. and de Walle, R.V., An integrated approach for device
independent publication of complex multimedia documents. In: Internet and Multimedia Systems
and Applications, (2005), 347–352
Jansen, J. and Bulterman, D.C., SMIL State: an architecture and implementation for adaptive timebased
web applications. Multimedia Tools and Applications 43, (2009), 203–224, DOI
1007/s11042-009-0270-3
Koch N., Pigerl M., Zhang G. and Morozova T., Patterns for the Model-Based Development of
RIAs. In: International Conference on Web Engineering (ICWE), Springer LNCS 5648, San
Sebastián, Spain, (2009), 283-291.
Kuijk, F., Guimara˜es, R.L., Cesar, P. and Bulterman, D.C., Adding dynamic visual manipulations
to declarative multimedia documents. In: 9th ACM Symposium on Document engineering,
DocEng ’09, ACM, NY, USA, (2009), 149–152, DOI 10.1145/1600193.1600227
LaLomia, M. and Sidowski, J., Measurements of computer satisfaction, literacy, and aptitudes: A
review. International Journal of Human-Computer Interaction 2(3), (1990), 231–253
Lalomia, M. and Spitzberg, B., Preliminary Development of a Model and Measure of Computer-
Mediated Communication (CMC) Competence. Journal of Computer-Mediated Communication,
, (2006), 629–666, doi:10.1111/j.1083-6101.2006.00030.
Linaje M., Sanchez-Figueroa, F. and Preciado, J., RUX-Method temporal schema specification.
Available at: https://s3-eu-west-1.amazonaws.com/homeria/RUX/RUXTemporalPresent.png
Linaje, M., Preciado, J.C., Morales-Chaparro, R., Rodriguez-Echeverria, R. and Sanchez-Figueroa,
F., Automatic Generation of RIAs Using RUX-Tool and Webratio. In: 9th International
Conference on Web Engineering, ICWE ’9, Springer-Verlag, Berlin, Heidelberg, (2009), 501–504,
DOI 10.1007/978-3-642-02818-2 48
Linaje, M., Preciado, J.C. and Sanchez-Figueroa, F., Domain-specific model for designing rich
internet application user interfaces. In: V. Lopez Jaquero, F. Montero Simarro, J.P. Molina Masso,
J. Vanderdonckt (eds.) Computer-Aided Design of User Interfaces VI, Springer London, (2009),
–306, DOI 10.1007/978-1-84882-206-1 27
Linaje, M., Preciado, J.C. and Sanchez-Figueroa, F., Engineering rich internet application user
interfaces over legacy web models. IEEE Internet Computing 11, (2007), 53–59 DOI
1109/MIC.2007.123
Linaje, M. and Preciado, J.C., Multi-device context-aware RIAs using a model-driven approach.
Journal of Universal Computer Science 16(15), (2010), 2038–2059
Linaje, M., Rux-method: modelado de interfaces de usuario web multidispositivo, multimedia,
interactivas y accesibles. Ph.D. thesis, Universidad de Extremadura, (2009)
Ma, H., Shin, K., Checking consistency in multimedia synchronization constraints. IEEE
Transactions on Multimedia, 6(4), (2004), 565–574
Meliá S., Gómez J., Pérez S. and Díaz O., A Model-Driven Development for GWT-Based Rich
Internet Applications with OOH4RIA. In: International Conference on Web Engineering (ICWE),
New York, USA, (2008), 13-23.
Meliá S., Gómez J., Pérez S. and Díaz O., Architectural and Technological Variability in Rich
Internet Applications. IEEE Internet Computing 14, 3 (2010), 24-32.
Microsoft: Expression Blend. Available at: http://www.microsoft.com/expression
Mikác, J., Roisin, C. and Le Duc, B., An export architecture for a multimedia authoring
environment. In: Proceeding of the eighth ACM symposium on Document engineering, DocEng
’08, ACM, NY, USA (2008), 28–31 DOI 10.1145/1410140.1410147
Montero, S., Fernandez, C., Dodero, J., Aedo, I. and Diaz, P., A design toolkit for hypermedia
applications based on Ariadne development method. In: R. Jacob, Q. Limbourg, J. Vanderdonckt
(eds.) Computer-Aided Design of User Interfaces IV, Springer Netherlands, (2005), 43–54, DOI
1007/1-4020-3304-4 4
Nielsen, J. and Hackos, J., Usability engineering, vol. 125184069. Academic press San Diego,
(1993)
Nouguier, E., Timesheets Scheduler. Available at: http://limsee3.gforge.inria.fr/publicsite/
timesheets/
Pellan, B. and Concolato, C., Authoring of scalable multimedia documents. MultimediaTools and
Applications 43, (2009), 225–252 DOI 10.1007/s11042-009-0268-x
Pihkala, K. and Vuorimaa, P., Nine methods to extend SMIL for multimedia applications.
Multimedia Tools and Applications 28, 51/67, (2006), DOI 10.1007/s11042-006-5120-y
Pimentel, M.d.G., Cattelan, R.G., Melo, E.L., Prado, A.F. and Teixeira, C.A.C., Ubiquitous enduser
live editing of interactive multimedia programs. In: 14th Brazilian Symposium on Multimedia
and the Web, WebMedia ’08, ACM, NY, USA, (2008), 123–129, DOI 10.1145/1666091.1666113
Pleuß, A., Botterweck, G. and Hußmann, H., Modeling advanced concepts of interactive
multimedia applications. In: Visual Languages and Human-Centric Computing, (2009), 31–38
DOI 10.1109/VLHCC.2009.5295305
Pleuß, A. and Hußmann, H., Integrating authoring tools into model-driven development of
interactive multimedia applications. In: 12th Int. Conf. on Human-computer interaction: interaction
design and usability, HCI’07, Springer-Verlag, Berlin, Heidelberg, (2007), 1168–1177
Pleuß, A., MML: a language for modeling interactive multimedia applications. In: 7th IEEE
International Symposium on Multimedia, (2005), 9–17 DOI 10.1109/ISM.2005.80
Preciado, J.C., Linaje, M., Sanchez, F. and Comai, S., Necessity of methodologies to model rich
internet applications. In: 7th IEEE International Symposium on Web Site Evolution, IEEE
Computer Society, Washington, DC, USA, (2005), 7–13, DOI 10.1109/WSE.2005.10
Rodrigues, L., Antonacci, M., Rodrigues, R., Muchaluat-Saade, D. and Soares, L., Improving
SMIL with NCM Facilities. Multimedia Tools and Applications 16, (2002), 29–54 DOI
1023/A:1013289601682
Santanchè, A., Medeiros, C. and Pastorello, G., User-author centered multimedia building blocks.
Multimedia systems 12(4), (2007), 403–421
Sauer, S. and Engels, G., Extending UML for modeling of multimedia applications. In: IEEE
Symposium on Visual Languages, IEEE Computer Society, Washington, DC, USA, (1999), 80–
,
Scherp, A., Canonical processes for creating personalized semantically rich multimedia
presentations. Multimedia Systems 14(6), (2008), 415–425
Schwinger, W., Retschitzegger, W., Schauerhuber, A., Kappel, G., Wimmer, M., Pr¨oll,B., Castro,
C.C., Casteleyn, S., Troyer, O.D., Fraternali, P., et al., A survey on web modeling approaches for
ubiquitous web applications. International Journal of Web Information Systems 4(3), (2008), 234–
Silva, H.V.O., Rodrigues, R.F., Soares, L.F.G. and Muchaluat Saade, D.C., NCL 2.0: integrating
new concepts to XML modular languages. In: Proceedings of the 2004 ACM symposium on
Document engineering, ACM, NY, USA (2004), 188–197, DOI 10.1145/1030397.1030433
Specht, G. and Zoller, P., HMT: Modeling Temporal Aspects in Hypermedia Applications. In: 1st
Int. Conf. on Web-Age Information Management, WAIM ’00, Springer-Verlag, London, UK,
(2000), 259–270
Sung, M. and Lee, D., A collaborative multimedia authoring system. In: M. Li, X.H. Sun,Q. Deng,
J. Ni (eds.) Grid and Cooperative Computing, Lecture Notes in Computer Science, vol. 3033,
Springer Berlin / Heidelberg, (2004), 311–318 DOI 10.1007/9783-540-24680-0 54
Thompson, S., King, P. and Cameron, H., Modelling reactive multimedia: Design and authoring.
Multimedia Tools and Applications 27, (2005), 23–52 DOI 10.1007/s11042005-2713-9
Toffetti, G., Comai, S., Preciado, J.C. and Trigueros, M.L., State-of-the-art and trends in the
systematic development of rich internet applications. Journal of Web Engineering 10(1), (2011),
–86
Vuorimaa, P., Timesheets JavaScript Engine. Available http://www.tml.tkk.fi/~pv/timesheets/
W3C: SMIL 3 (2008). Available at: http://www.w3.org/TR/SMIL3/.
W3C: SMIL Timesheets 1.0 (2012). Available at: https://www.w3.org/TR/timesheets/.
W3C: Synchronized Multimedia Authoring Tools. Available at:
http://www.w3.org/AudioVideo/#Authoring.
Willrich, R., De Saqui-Sannes, P., Sénac, P. and Diaz, M., Multimedia authoring with hierarchical
timed stream Petri nets and java. Multimedia Tools and Applications 16, (2002), 7–27 DOI
1023/A:1013233517612
Yang, C. C., Wang, Y. C. and Tien, C. W., Synchronization modeling and its application for
SMIL2. 0 presentations. Journal of Systems and Software, 80(7), (2007), 1142-1155
Yang, C.C., Chu, C.K. and WANG, Y.C., Extension of Timeline-based Editing for
Nondeterministic Temporal Behavior in SMIL2.0 Authoring. Journal of information science and
engineering 24(5), (2008), 1377–1395
Yang, C.C. and Yang, Y.Z., SMILAuthor: an authoring system for SMIL-based multimedia
presentations. Multimedia Tools and Applications 21, (2003), 243–260 DOI
1023/A:1025770817293
