MEASURES AND TECHNIQUES FOR EFFORT ESTIMATION OF WEB APPLICATIONS: AN EMPIRICAL STUDY BASED ON A SINGLE-COMPANY DATASET
Keywords:
Web applications, Size measures, Effort estimation, Empirical validationAbstract
Effort estimation is a key management activity which goes on throughout a software project being fundamental for accurate project planning and for allocating resources adequately. Thus, it is important to identify techniques and measures that can support such project management activity during the development of Web applications. To this aim, empirical investigations should be performed using data coming from the industrial world. To address this issue, this paper reports on an empirical study based on data from 15 Web applications developed by an Italian software company. The objective of the study was two-fold. The first goal was to verify whether or not some size measures were good indicators of the effort spent to develop the Web applications taken into account. The second goal was to compare the effectiveness of some techniques to establish the relationships between the employed size measures and the development effort of the Web applications. The measures were organized in two sets, where the first one included some length measures while the second one consisted of the nine components which are used to estimate the Web Objects measure. The techniques taken into account were Stepwise Regression, Case- Based Reasoning, and Regression Tree. The results indicated that both the sets of size measures were good indicators of the effort for the analyzed dataset. Furthermore, the analysis also revealed that the first set presented significantly superior performance than the second set when using Stepwise Regression. No significant differences between the two sets of size measures were highlighted when using Case-Based Reasoning and Regression Tree.
Downloads
References
A. Aamodt, E. Plaza, “Case-Based Reasoning: Foundational Issues, Methodological Variations,
and System Approaches”, AI Communication, IOS Press, 7(1), 1994, pp. 39-59.
S.M. Abrahão, O. Pastor, “Measuring the Functional Size of Web applications”, in International
Journal of Web Engineering and Technology, 1(1), 2003, pp. 5-16.
S.M. Abrahão, Geert Poels, O. Pastor, “Evaluating a Functional Size Measurement Method for
Web Applications: An Empirical Analysis”, in Proceedings of Tenth International Software
Metrics Symposium (METRICS’04), Chicago, Illinois, USA, 20004, pp. 358-369.
A. Abran, T. M. Khosgoftaar, A. Idri, “Fuzzy Analogy: A New Approach for Software Cost
Estimation“, in Proceedings of the International Workshop on Software Measurement (IWSM’01),
Montreal, Canada, 2001, pp. 93-101.
A.J. Albrecht, "Measuring Application Development Productivity," in Proceedings of the Joint
SHARE/GUIDE/IBM Application Development Symposium, Monterey, CA, 1979, pp. 83-92.
L. Baresi, S. Morasca, P: Paolini, “Estimating the Design Effort of Web Applications,” in Proc. of
the 9th International Software Metrics Symposium, Sydney, Australia, 2003, pp. 62-72.
V.R. Basili, L.C. Briand, W.L. Melo, “A Validation of Object-Oriented Design Metrics as Quality
Indicators,” IEEE Transactions on Software Engineering, 22(10), 1996, pp. 751-761.
V. R. Basili, F. Shull, F. Lanubile “Building Knowledge through Families of Experiments“ IEEE
Transactions on Software Engineering, 25(4), August 1999, pp. 456-473,.
B. Bohem, Software Engineering Economics, Prentice-Hall, Englewood Cloffs, NJ, 1981.
L. Briand, K. E. Emam, D. Surmann, I. Wiekzorek, K. Maxwell “An Assessment and Comparison
of Common Software Cost Estimation Modeling Techniques,” in Proceedings of International
Conference on Software Engineering, (ICSE’99), Los Angeles, USA, 1999.
L. Briand, T. Langley, I. Wiekzorek, “A Replicated Assessment and Comparison of Common
Software Cost Modeling Techniques”, International Software Engineering Research Network
Technical Report ISERN-99-15.
L. Briand, I. Wieczorek. Software Resource Estimation. Encyclopedia of Software Engineering.
Volume 2. P-Z (2nd ed.), Marciniak, John J. (ed.) New York: John Wiley & Sons, 2002, pp. 1160-
L. Briand, J. Wüst, “Modeling Development Effort in Object-Oriented Systems Using Design
Properties. IEEE Transactions on Software Engineering 27(11), 2001, pp. 963-986.
A. Comrey, “A First Course on Factor Analysis”, London: Academic Press, 1973.
J. Cohen, Statistical Power Analysis for the Behavioral Science, Lawrence Erlbaum Hillsdale,
New Jersey, 1988.
J. Conallen, Building Web Applications with UML, Addison-Wesley Object Technology Series,
D. Conte, H.E. Dunsmore, V.Y. Shen, Software Engineering Metrics and Models, The
Benjamin/Cummings Publishing Company, Inc., 1986.
R.D. Cook, “Detection of influential observations in linear regression, Technometrics, 19, 1977,
pp. 15-18.
G. Costagliola, S. Di Martino, F. Ferrucci, C. Gravino, G. Tortora, G. Vitiello, “Effort Estimation
Modeling Techniques: A Case Study for Web Applications”, in ACM Proceedings of the 6th
International Conference on Web Engineering (ICWE 2006), 2006, pp. 9-16.
G. Costagliola, F. Ferrucci, C. Gravino, G. Tortora, G. Vitello, “A COSMIC-FFP Based Method
to Estimate Web Application Development Effort”, in LNCS 3140, N. Koch, P. Fraternali, and M.
Wirsing (Eds.): International Conference on Web Engineering (ICWE’04), Monaco, Germany,
, pp.161-165.
T. Dybå, N. B. Moe, E. M. Mikkelsen, ”An Empirical Investigation on Factors Affecting Software
Developer Acceptance and Utilization of Electronic Process Guides”, in Proceedings of IEEE
International Symposium on Software Metrics (METRICS'04), 2004, pp. 220-231.
T. Dybå, “An Empirical Investigation of the Key Factors for Success in Software Process
Improvement”, IEEE Transactions on Software Engineering, 31(5), 2005, pp. 410-424.
K.E. Emam, “A Primer on Object-Oriented Measurement”, in Proceedings of IEEE International
Software Metrics Symposium (METRICS’01), London, 2001, pp. 185-188.
R. L. Gorsuch, “Factor Analysis”, . 2nd ed., Hillsdale: Lawrence Erlbaum Associates, 1983.
International Function Point Users Group: “Function Point Counting Practices Manual,” Release
2.1, 2004.
G. Kadoba, M. Cartwright, L. Chen, M. Shepperd, “Experiences Using Case-Based Reasoning to
Predict Software Project Effort”, in Proceedings of EASE 2000 Conference, Keele, UK, 2000.
V, By Kampenes, T. Dybå, J. E. Hannay, D. I.K. Sjøberg, “A Systematic Review of Effect Size in
Software Engineering Experiments”, Information and Software Technology 4(11-12), 2007,
pp.1073-1086.
B. A. Kitchenham, “A Procedure for Analyzing Unbalanced Datasets”, IEEE Transactions on
Software Engineering, 24(4), 1998, pp. 278-301.
A. Kitchenham, S. L. Pfleeger, L. M. Pickard “Case Studies for Method and Tool Evaluation”,
IEEE Software , 12(4), 1995, pp. 52-62.
A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. El Emam, J.
Rosenberg “Preliminary Guidelines for Empirical Research in Software Engineering”, IEEE
Transactions on Software Engineering, 28(8), 2002, pp. 721–734.
G. Kadoba, M. Shepperd, “Using Simulation to Evaluate Predictions Systems”, in Proceedings of
International Software Metrics Symposium (METRICS’01), England, 2001, pp. 349-358.
B. A. Kitchenham, L. M. Pickard, S. G. MacDonell, M. J. Shepperd, “What accuracy statistics
really measure”, IEE Proceedings – Software, 148(3), 2001, pp. 81-85.
S. Kunkel, U. Rosenqvist, R. Westerling, “The structure of quality systems is important to the
process and outcome, an empirical study of 386 hospital departments in Sweden”, BMC Health
Serv Res. 7 (1), 2007, pp. 104.
R. Jeffery, J. Stathis, “Function point sizing: Structure, validity and applicability”, Empirical
Software Engineering, 1(1), 1996, pp. 11-30.
K. Maxwell, “Applied Statistics for Software Managers”. Software Quality Institute Series,
Prentice Hall, 2002.
Measurement in Software Engineering, Web site
http://www2.umassd.edu/SWPI/ProcessBibliography/bib-measurement.html. Last visited on May
, 2006.
E. Mendes, S. Counsell, N. Mosley, “Comparison of Web Size Measures for Predicting Web
Design and Authoring Effort”, IEE Proceedings-Software 149(3), 2002, pp. 86-92.
E. Mendes, S. Counsell, N. Mosley, “Web Metrics – Estimating Design and Authoring Effort”,
IEEE Multimedia, Special Issue on Web Engineering, 8(1), 2001, pp. 50-57.
E. Mendes, S. Counsell, N. Mosley, C. Triggs, I. Watson, “A Comparative Study of Cost
Estimation Models for Web Hypermedia Applications”, Empirical Software Engineering 8(2),
, pp. 163-196.
E. Mendes, S. Counsell, N. Mosley, “Early Web Size Measures and Effort Prediction for Web
Costimation”, in Proceedings of International Software Metrics Symposium (METRICS’03),
Sydney, Australia, 2003, pp. 18-39.
E. Mendes, S. Counsell, N. Mosley, “A Replicated Assessment of the Use of Adaptation Rules to
Improve Web Cost Estimation”, in Proceedings of the International Symposium on Empirical
Software Engineering (ISESE’03), Rome, Italy, 2003, pp. 100.
E. Mendes, S. Di Martino, F. Ferrucci, C. Gravino. “Effort Estimation: How Valuable is it for a
Web company to Use a Cross-company Data Set, Compared to Using Its Own Single-company
Data Set?” In ACM Proceedings of the 6th International World Wide Web Conference
(WWW2007), Banff, Canada, 10-13 2007, pp 963 – 972.
E. Mendes, N. Mosley, “Towards a Taxonomy of Hypermedia and Web Application Size
Metrics”, in Proceedings of International Conference of Web Engineering (ICWE 2005), Sydney,
Australia, 2005, pp. 110-123.
E. Mendes, B. Kitchenham, “A Comparison of Cross-company and Within-company Effort
Estimation Models for Web Applications”, in Proceedings of Conference on Evaluation &
Assessment in Software Engineering (EASE 2004), Edinburgh, Scotland, 2004, pp. 47-55.
E. Mendes, B. Kitchenham, “Further Comparison of Cross-Company and Within-Company Effort
Estimation Models for Web Applications”, in Proceedings of the International Symposium on
Software Metrics (METRICS'04), Chicago, IL, USA, 2004, pp. 348-357.
E. Mendes, N. Mosley, “Further Investigation into the Use of CBR and Stepwise Regression to
Predict Development Effort for Web Hypermedia Applications”, in Proc. of International
Symposium on Empirical Software Engineering (IESE’02), Japan, 2002, pp. 79-90.
L. A. Merkle, C. S. Layne, J. J. Bloomberg, J. J. Zhang “Using factor analysis to identify
neuromuscular synergies during treadmill walking”, Journal of Neuroscience Methods, 82(2),
, pp. 207-214.
D. Montgomery, E. Peck, G. Vining, Introduction to Linear Regression Analysis, John Wiley &
Sons, Inc., 3 Ed., 2001.
M. Morisio, I. Stamelos, V. Spahos, D. Romano, “Measuring Functionality and Productivity in
Web-based applications: a Case Study”, in Proceedings of the International Software Metrics
Symposium (METRICS’99), Boca Raton, 1999, pp. 111-118.
M. Morisio, D. Romano, I. Stamelos, “Quality, Productivity, and Learning in Framework-Based
Development: An Exploratory Case Study”, in IEEE Transactions on Software Engineering,
(9), 2002, pp. 876-888.
J. C. Munson, T. M. Khoshgoftaar, “The Detection of Fault-Prone Programs”, Transactions on
Software Engineering 18(5), 1992. pp. 423-433.
I. Myrtveit, E. Stensrud, “A Controlled Experiment to Assess the Benefits of Estimating with
Analogy and Regression Models”, IEEE Transactions on Software Engineering, 25(4), 1999, pp.
-525.
D. Perry, A. Porter, L. Votta, “Empirical Studies of Software Engineering: A Roadmap”, The
Future of Software Engineering, Ed: Anthony Finkelstein, ACM Press, 2000, pp. 345-355.
D. Reifer, “Web-Development: Estimating Quick-Time-to-Market Software”, IEEE Software,
(8), 2000, pp. 57-64.
D. Reifer, “Web Objects Counting Conventions”, Reifer Consultants, Mar. 2001. Available at:
http://www.reifer.com/download.html.
T. Rollo, “Sizing E-Commerce”, in Proceedings of the ACOSM 2000 - Australian Conference on
Software Measurement, Sydney, 2000.
M. Ruhe, R. Jeffery, I. Wieczorek, “Cost Estimation for Web applications”, in the Proceedings of
th International Conference on Software Engineering (ICSE’03), Oregon USA, 2003, pp. 285 –
M. Ruhe, R. Jeffery, I. Wieczorek, “Using Web Objects for Estimating Software Development
Effort for Web Applications”, in Proceedings of International Software Metrics Symposium
(METRICS’03), Sydney, Australia, 2003, pp. 30-37.
M. Shepperd, C. Schofield, “Estimating Software Project Effort using Analogies”, in IEEE
Transactions on Software Engineering, 23(11), 2000, pp. 736-743.
M. Shepperd, C. Schofield, B. Kitchenham, “Effort Estimation using Analogy”, in Proceedings of
International Conference on Software Engineering (ICSE’96), Berlin Germany, 1996, pp.170-178.
E. Stensrud, T. Foss, B. Kitchenham, I. Myrtveit, ”A Further Empirical Investigation of the
Relationship between MRE and Project Size”, Empirical Software Engineering, 8(2), 2003, pp.
-161.
P. Umbers, G Miles, “Resource Estimation for Web Applications”, in Proceedings of tenth
International Software Metrics Symposium (METRICS’04), Chicago, Illinois USA, 2004, pp. 370-
F. Walkerden, R. Jeffery, “An Empirical Study of Analogy-based Software Effort Estimation”,
Empirical Software Engineering, 4(2), 1999, pp. 135-158.
C.Wohlin, P. Runeson, M. Host, M. C. Ohlsson, B. Regnell, A.Wesslen, Experimentation in
Software Engineering - An Introduction. Kluwer, 2000.
M. V. Zelkowitz, D. R. Wallace, "Experimental Models for Validating Technology", Computer,
(5), 1998, pp. 23-31.
