MEASURES AND TECHNIQUES FOR EFFORT ESTIMATION OF WEB APPLICATIONS: AN EMPIRICAL STUDY BASED ON A SINGLE-COMPANY DATASET

Authors

  • SERGIO DI MARTINO Dipartimento di Matematica e Informatica, University of Salerno, Italy
  • FILOMENA FERRUCCI Dipartimento di Matematica e Informatica, University of Salerno, Italy
  • CARMINE GRAVINO Dipartimento di Matematica e Informatica, University of Salerno, Italy
  • EMILIA MENDES The University of Auckland, Private Bag 92019

Keywords:

Web applications, Size measures, Effort estimation, Empirical validation

Abstract

Effort estimation is a key management activity which goes on throughout a software project being fundamental for accurate project planning and for allocating resources adequately. Thus, it is important to identify techniques and measures that can support such project management activity during the development of Web applications. To this aim, empirical investigations should be performed using data coming from the industrial world. To address this issue, this paper reports on an empirical study based on data from 15 Web applications developed by an Italian software company. The objective of the study was two-fold. The first goal was to verify whether or not some size measures were good indicators of the effort spent to develop the Web applications taken into account. The second goal was to compare the effectiveness of some techniques to establish the relationships between the employed size measures and the development effort of the Web applications. The measures were organized in two sets, where the first one included some length measures while the second one consisted of the nine components which are used to estimate the Web Objects measure. The techniques taken into account were Stepwise Regression, Case- Based Reasoning, and Regression Tree. The results indicated that both the sets of size measures were good indicators of the effort for the analyzed dataset. Furthermore, the analysis also revealed that the first set presented significantly superior performance than the second set when using Stepwise Regression. No significant differences between the two sets of size measures were highlighted when using Case-Based Reasoning and Regression Tree.

 

Downloads

Download data is not yet available.

References

A. Aamodt, E. Plaza, “Case-Based Reasoning: Foundational Issues, Methodological Variations,

and System Approaches”, AI Communication, IOS Press, 7(1), 1994, pp. 39-59.

S.M. Abrahão, O. Pastor, “Measuring the Functional Size of Web applications”, in International

Journal of Web Engineering and Technology, 1(1), 2003, pp. 5-16.

S.M. Abrahão, Geert Poels, O. Pastor, “Evaluating a Functional Size Measurement Method for

Web Applications: An Empirical Analysis”, in Proceedings of Tenth International Software

Metrics Symposium (METRICS’04), Chicago, Illinois, USA, 20004, pp. 358-369.

A. Abran, T. M. Khosgoftaar, A. Idri, “Fuzzy Analogy: A New Approach for Software Cost

Estimation“, in Proceedings of the International Workshop on Software Measurement (IWSM’01),

Montreal, Canada, 2001, pp. 93-101.

A.J. Albrecht, "Measuring Application Development Productivity," in Proceedings of the Joint

SHARE/GUIDE/IBM Application Development Symposium, Monterey, CA, 1979, pp. 83-92.

L. Baresi, S. Morasca, P: Paolini, “Estimating the Design Effort of Web Applications,” in Proc. of

the 9th International Software Metrics Symposium, Sydney, Australia, 2003, pp. 62-72.

V.R. Basili, L.C. Briand, W.L. Melo, “A Validation of Object-Oriented Design Metrics as Quality

Indicators,” IEEE Transactions on Software Engineering, 22(10), 1996, pp. 751-761.

V. R. Basili, F. Shull, F. Lanubile “Building Knowledge through Families of Experiments“ IEEE

Transactions on Software Engineering, 25(4), August 1999, pp. 456-473,.

B. Bohem, Software Engineering Economics, Prentice-Hall, Englewood Cloffs, NJ, 1981.

L. Briand, K. E. Emam, D. Surmann, I. Wiekzorek, K. Maxwell “An Assessment and Comparison

of Common Software Cost Estimation Modeling Techniques,” in Proceedings of International

Conference on Software Engineering, (ICSE’99), Los Angeles, USA, 1999.

L. Briand, T. Langley, I. Wiekzorek, “A Replicated Assessment and Comparison of Common

Software Cost Modeling Techniques”, International Software Engineering Research Network

Technical Report ISERN-99-15.

L. Briand, I. Wieczorek. Software Resource Estimation. Encyclopedia of Software Engineering.

Volume 2. P-Z (2nd ed.), Marciniak, John J. (ed.) New York: John Wiley & Sons, 2002, pp. 1160-

L. Briand, J. Wüst, “Modeling Development Effort in Object-Oriented Systems Using Design

Properties. IEEE Transactions on Software Engineering 27(11), 2001, pp. 963-986.

A. Comrey, “A First Course on Factor Analysis”, London: Academic Press, 1973.

J. Cohen, Statistical Power Analysis for the Behavioral Science, Lawrence Erlbaum Hillsdale,

New Jersey, 1988.

J. Conallen, Building Web Applications with UML, Addison-Wesley Object Technology Series,

D. Conte, H.E. Dunsmore, V.Y. Shen, Software Engineering Metrics and Models, The

Benjamin/Cummings Publishing Company, Inc., 1986.

R.D. Cook, “Detection of influential observations in linear regression, Technometrics, 19, 1977,

pp. 15-18.

G. Costagliola, S. Di Martino, F. Ferrucci, C. Gravino, G. Tortora, G. Vitiello, “Effort Estimation

Modeling Techniques: A Case Study for Web Applications”, in ACM Proceedings of the 6th

International Conference on Web Engineering (ICWE 2006), 2006, pp. 9-16.

G. Costagliola, F. Ferrucci, C. Gravino, G. Tortora, G. Vitello, “A COSMIC-FFP Based Method

to Estimate Web Application Development Effort”, in LNCS 3140, N. Koch, P. Fraternali, and M.

Wirsing (Eds.): International Conference on Web Engineering (ICWE’04), Monaco, Germany,

, pp.161-165.

T. Dybå, N. B. Moe, E. M. Mikkelsen, ”An Empirical Investigation on Factors Affecting Software

Developer Acceptance and Utilization of Electronic Process Guides”, in Proceedings of IEEE

International Symposium on Software Metrics (METRICS'04), 2004, pp. 220-231.

T. Dybå, “An Empirical Investigation of the Key Factors for Success in Software Process

Improvement”, IEEE Transactions on Software Engineering, 31(5), 2005, pp. 410-424.

K.E. Emam, “A Primer on Object-Oriented Measurement”, in Proceedings of IEEE International

Software Metrics Symposium (METRICS’01), London, 2001, pp. 185-188.

R. L. Gorsuch, “Factor Analysis”, . 2nd ed., Hillsdale: Lawrence Erlbaum Associates, 1983.

International Function Point Users Group: “Function Point Counting Practices Manual,” Release

2.1, 2004.

G. Kadoba, M. Cartwright, L. Chen, M. Shepperd, “Experiences Using Case-Based Reasoning to

Predict Software Project Effort”, in Proceedings of EASE 2000 Conference, Keele, UK, 2000.

V, By Kampenes, T. Dybå, J. E. Hannay, D. I.K. Sjøberg, “A Systematic Review of Effect Size in

Software Engineering Experiments”, Information and Software Technology 4(11-12), 2007,

pp.1073-1086.

B. A. Kitchenham, “A Procedure for Analyzing Unbalanced Datasets”, IEEE Transactions on

Software Engineering, 24(4), 1998, pp. 278-301.

A. Kitchenham, S. L. Pfleeger, L. M. Pickard “Case Studies for Method and Tool Evaluation”,

IEEE Software , 12(4), 1995, pp. 52-62.

A. Kitchenham, S. L. Pfleeger, L. M. Pickard, P. W. Jones, D. C. Hoaglin, K. El Emam, J.

Rosenberg “Preliminary Guidelines for Empirical Research in Software Engineering”, IEEE

Transactions on Software Engineering, 28(8), 2002, pp. 721–734.

G. Kadoba, M. Shepperd, “Using Simulation to Evaluate Predictions Systems”, in Proceedings of

International Software Metrics Symposium (METRICS’01), England, 2001, pp. 349-358.

B. A. Kitchenham, L. M. Pickard, S. G. MacDonell, M. J. Shepperd, “What accuracy statistics

really measure”, IEE Proceedings – Software, 148(3), 2001, pp. 81-85.

S. Kunkel, U. Rosenqvist, R. Westerling, “The structure of quality systems is important to the

process and outcome, an empirical study of 386 hospital departments in Sweden”, BMC Health

Serv Res. 7 (1), 2007, pp. 104.

R. Jeffery, J. Stathis, “Function point sizing: Structure, validity and applicability”, Empirical

Software Engineering, 1(1), 1996, pp. 11-30.

K. Maxwell, “Applied Statistics for Software Managers”. Software Quality Institute Series,

Prentice Hall, 2002.

Measurement in Software Engineering, Web site

http://www2.umassd.edu/SWPI/ProcessBibliography/bib-measurement.html. Last visited on May

, 2006.

E. Mendes, S. Counsell, N. Mosley, “Comparison of Web Size Measures for Predicting Web

Design and Authoring Effort”, IEE Proceedings-Software 149(3), 2002, pp. 86-92.

E. Mendes, S. Counsell, N. Mosley, “Web Metrics – Estimating Design and Authoring Effort”,

IEEE Multimedia, Special Issue on Web Engineering, 8(1), 2001, pp. 50-57.

E. Mendes, S. Counsell, N. Mosley, C. Triggs, I. Watson, “A Comparative Study of Cost

Estimation Models for Web Hypermedia Applications”, Empirical Software Engineering 8(2),

, pp. 163-196.

E. Mendes, S. Counsell, N. Mosley, “Early Web Size Measures and Effort Prediction for Web

Costimation”, in Proceedings of International Software Metrics Symposium (METRICS’03),

Sydney, Australia, 2003, pp. 18-39.

E. Mendes, S. Counsell, N. Mosley, “A Replicated Assessment of the Use of Adaptation Rules to

Improve Web Cost Estimation”, in Proceedings of the International Symposium on Empirical

Software Engineering (ISESE’03), Rome, Italy, 2003, pp. 100.

E. Mendes, S. Di Martino, F. Ferrucci, C. Gravino. “Effort Estimation: How Valuable is it for a

Web company to Use a Cross-company Data Set, Compared to Using Its Own Single-company

Data Set?” In ACM Proceedings of the 6th International World Wide Web Conference

(WWW2007), Banff, Canada, 10-13 2007, pp 963 – 972.

E. Mendes, N. Mosley, “Towards a Taxonomy of Hypermedia and Web Application Size

Metrics”, in Proceedings of International Conference of Web Engineering (ICWE 2005), Sydney,

Australia, 2005, pp. 110-123.

E. Mendes, B. Kitchenham, “A Comparison of Cross-company and Within-company Effort

Estimation Models for Web Applications”, in Proceedings of Conference on Evaluation &

Assessment in Software Engineering (EASE 2004), Edinburgh, Scotland, 2004, pp. 47-55.

E. Mendes, B. Kitchenham, “Further Comparison of Cross-Company and Within-Company Effort

Estimation Models for Web Applications”, in Proceedings of the International Symposium on

Software Metrics (METRICS'04), Chicago, IL, USA, 2004, pp. 348-357.

E. Mendes, N. Mosley, “Further Investigation into the Use of CBR and Stepwise Regression to

Predict Development Effort for Web Hypermedia Applications”, in Proc. of International

Symposium on Empirical Software Engineering (IESE’02), Japan, 2002, pp. 79-90.

L. A. Merkle, C. S. Layne, J. J. Bloomberg, J. J. Zhang “Using factor analysis to identify

neuromuscular synergies during treadmill walking”, Journal of Neuroscience Methods, 82(2),

, pp. 207-214.

D. Montgomery, E. Peck, G. Vining, Introduction to Linear Regression Analysis, John Wiley &

Sons, Inc., 3 Ed., 2001.

M. Morisio, I. Stamelos, V. Spahos, D. Romano, “Measuring Functionality and Productivity in

Web-based applications: a Case Study”, in Proceedings of the International Software Metrics

Symposium (METRICS’99), Boca Raton, 1999, pp. 111-118.

M. Morisio, D. Romano, I. Stamelos, “Quality, Productivity, and Learning in Framework-Based

Development: An Exploratory Case Study”, in IEEE Transactions on Software Engineering,

(9), 2002, pp. 876-888.

J. C. Munson, T. M. Khoshgoftaar, “The Detection of Fault-Prone Programs”, Transactions on

Software Engineering 18(5), 1992. pp. 423-433.

I. Myrtveit, E. Stensrud, “A Controlled Experiment to Assess the Benefits of Estimating with

Analogy and Regression Models”, IEEE Transactions on Software Engineering, 25(4), 1999, pp.

-525.

D. Perry, A. Porter, L. Votta, “Empirical Studies of Software Engineering: A Roadmap”, The

Future of Software Engineering, Ed: Anthony Finkelstein, ACM Press, 2000, pp. 345-355.

D. Reifer, “Web-Development: Estimating Quick-Time-to-Market Software”, IEEE Software,

(8), 2000, pp. 57-64.

D. Reifer, “Web Objects Counting Conventions”, Reifer Consultants, Mar. 2001. Available at:

http://www.reifer.com/download.html.

T. Rollo, “Sizing E-Commerce”, in Proceedings of the ACOSM 2000 - Australian Conference on

Software Measurement, Sydney, 2000.

M. Ruhe, R. Jeffery, I. Wieczorek, “Cost Estimation for Web applications”, in the Proceedings of

th International Conference on Software Engineering (ICSE’03), Oregon USA, 2003, pp. 285 –

M. Ruhe, R. Jeffery, I. Wieczorek, “Using Web Objects for Estimating Software Development

Effort for Web Applications”, in Proceedings of International Software Metrics Symposium

(METRICS’03), Sydney, Australia, 2003, pp. 30-37.

M. Shepperd, C. Schofield, “Estimating Software Project Effort using Analogies”, in IEEE

Transactions on Software Engineering, 23(11), 2000, pp. 736-743.

M. Shepperd, C. Schofield, B. Kitchenham, “Effort Estimation using Analogy”, in Proceedings of

International Conference on Software Engineering (ICSE’96), Berlin Germany, 1996, pp.170-178.

E. Stensrud, T. Foss, B. Kitchenham, I. Myrtveit, ”A Further Empirical Investigation of the

Relationship between MRE and Project Size”, Empirical Software Engineering, 8(2), 2003, pp.

-161.

P. Umbers, G Miles, “Resource Estimation for Web Applications”, in Proceedings of tenth

International Software Metrics Symposium (METRICS’04), Chicago, Illinois USA, 2004, pp. 370-

F. Walkerden, R. Jeffery, “An Empirical Study of Analogy-based Software Effort Estimation”,

Empirical Software Engineering, 4(2), 1999, pp. 135-158.

C.Wohlin, P. Runeson, M. Host, M. C. Ohlsson, B. Regnell, A.Wesslen, Experimentation in

Software Engineering - An Introduction. Kluwer, 2000.

M. V. Zelkowitz, D. R. Wallace, "Experimental Models for Validating Technology", Computer,

(5), 1998, pp. 23-31.

Downloads

Published

2009-10-30

Issue

Section

Articles