• MARIO BARRENECHEA Department of Computer Science, University of Colorado Boulder, 430 UCB Boulder, Colorado 80309-0430, USA
  • SAHAR JAMBI Department of Computer Science, University of Colorado Boulder, 430 UCB Boulder, Colorado 80309-0430, USA
  • AHMET ARIF AYDIN Department of Computer Science, University of Colorado Boulder, 430 UCB Boulder, Colorado 80309-0430, USA
  • MAZIN HAKEEM Department of Computer Science, University of Colorado Boulder, 430 UCB Boulder, Colorado 80309-0430, USA
  • KENNETH M. ANDERSON Department of Computer Science, University of Colorado Boulder, 430 UCB Boulder, Colorado 80309-0430, USA


data-intensive systems, web applications, design issues, crisis informatics


Web-based data analysis environments are powerful platforms for exploring large data sets. To ensure that these environments meet the needs of analysts, a human-centered design perspective is needed. Interfaces to these platforms should provide exible search, support user-generated content, and enable collaboration. We report on our eorts to design and develop a web interface for a custom analytics platform|EPIC Analyze| which provides interactive search over large Twitter data sets collected during crisis events. We performed seven think-aloud sessions with researchers who regularly analyze crisis data sets and compiled their feedback. They identied a need for a \big picture" view of an event, exible querying capabilities, and user-dened coding schemes. Adding these features allowed EPIC Analyze to meet the needs of these analysts and enable exploratory research on crisis data. In performing this work, we identied an opportunity to migrate the software architecture of EPIC Analyze to one based on microservices. We report on the lessons learned in performing this migration and the impact it had on EPIC Analyze's capabilities. We also re ect on the benets a microservices approach can have on the design of data-intensive software systems like EPIC Analyze.


Download data is not yet available.


K. M. Anderson, Embrace the challenges: Software engineering in a big data world," in First

International Workshop on Big Data Software Engineering, Part of the 2015 International Con-

ference on Software Engineering, pp. 19{25, May 2015.

L. Palen, K. M. Anderson, G. Mark, J. Martin, D. Sicker, M. Palmer, and D. Grunwald, A

vision for technology-mediated support for public participation & assistance in mass emergencies

& disasters," in ACM-BCS Visions of Computer Science, April 2010. Article 8. 12 pages.

K. M. Anderson and A. Schram, Design and implementation of a data analytics infrastructure

in support of crisis informatics research (nier track)," in International Conference on Software

Engineering, pp. 844{847, May 2011.

K. M. Anderson, A. Schram, A. Alzabarah, and L. Palen, Architectural implications of social me-

dia analytics in support of crisis informatics research," IEEE Bulletin of the Technical Committee

on Data Engineering, vol. 36, pp. 13{20, September 2013.

A. Schram and K. M. Anderson, MySQL to NoSQL: Data modeling challenges in supporting scal-

ability," in Systems, Programming Languages, and Applications: Software for Humanity, pp. 191{

, October 2012.

K. M. Anderson, A. A. Aydin, M. Barrenechea, A. Cardenas, M. Hakeem, and S. Jambi, Design

challenges/solutions for environments supporting the analysis of social media data in crisis in-

formatics research," in Hawaii International Conference on System Sciences, pp. 163{172, IEEE,

January 2015.

D. Fisher, R. DeLine, M. Czerwinski, and S. Drucker, Interactions with big data analytics,"

Interactions, vol. 19, pp. 50{59, May + June 2012.

L. Palen, S. Vieweg, J. Sutton, S. Liu, and A. Hughes, Crisis informatics: Studying crisis in a

networked world," in Third International Conference on E-Social Science, October 2007. 10 pages.

A. Sarcevic, L. Palen, J. White, K. Starbird, M. Bagdouri, and K. M. Anderson, `Beacons of

hope' in decentralized coordination: Learning from on-the-ground medical twitterers during the

haiti earthquake," in Computer Supported Cooperative Work, pp. 47{56, February 2012.

A. L. Hughes, L. St. Denis, L. Palen, and K. M. Anderson, Online public communications by

police & re services during the 2012 hurricane sandy," in Human Factors in Computing Systems,

pp. 1505{1514, April 2014.

L. Huang, K. Starbird, M. Orand, S. Stanek, and H. Pedersen, Connected through crisis: Emo-

tional proximity and the spread of misinformation online," in Computer Supported Cooperative

Work, pp. 969{980, March 2015.

M. Barrenechea, K. M. Anderson, A. A. Aydin, M. Hakeem, and S. Jambi, Getting the query

right: User interface design of analysis platforms for crisis research," in International Conference

on Web Engineering, pp. 547{564, June 2015.

C. Hagar, Crisis informatics: Perspectives of trust|is social media a mixed blessing?," iSchool

Student Research Journal, vol. 2, no. 2, 2013.

K. Starbird and L. Palen, `Voluntweeters': Self-organizing by digital volunteers in times of crisis,"

in Human Factors in Computing Systems, pp. 1071{1080, May 2011.

J. White, L. Palen, and K. M. Anderson, Digital mobilization in disaster response: The work &

self-organization of on-line pet advocates in response to hurricane sandy," in Computer Supported

Cooperative Work and Social Computing, pp. 866{876, February 2014.

K. Starbird, L. Palen, A. Hughes, and S. Vieweg, Chatter on the red: What hazards threat

reveals about the social life of microblogged information," in Computer Supported Cooperative

Work, pp. 241{250, February 2010.

L. Palen, R. Soden, T. J. Anderson, and M. Barrenechea, Success and scale in a data-producing

organization: The socio-technical evolution of openstreetmap in response to humanitarian events,"

in Human Factors in Computing Systems, pp. 4113{4122, April 2015.

J. Bargas-Avila and K. Hornbk, Foci and blind spots in user experience research," Interactions,

vol. 19, pp. 24{25, November + December 2012.

C. McTaggart, Analysis and implementation of software tools to support research in crisis infor-

matics," Master's thesis, University of Colorado, 2012. 65 pages.

J. Laconich, J. Jose, F. Casati, and M. Marchese, Social spreadsheet," in Web Engineering

(F. Daniel, P. Dolog, and Q. Li, eds.), vol. 7977 of Lecture Notes in Computer Science, pp. 156{170,

Springer Berlin Heidelberg, 2013.

C. Lewis and J. Rieman, Task-centered User Interface Design: A Practical Introduction. Depart-

ment of Computer Science, University of Colorado, Boulder, 1993. 170 pages.

B. Shneiderman, The eyes have it: A task by data type taxonomy for information visualizations,"

in IEEE Symposium on Visual Languages, pp. 336{343, September 1996.

A. A. Aydin and K. M. Anderson, Incremental sorting for large dynamic data sets," in First

IEEE International Conference on Big Data Computing Service and Applications, pp. 170{175,

March + April 2015.

C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins, Pig latin: A not-so-foreign lan-

guage for data processing," in International Conference on the Management of Data, SIGMOD

'08, (New York, NY, USA), pp. 1099{1110, ACM, June 2008.

M. Fowler and J. Lewis, Microservices," March 2014.


I. Graham, Object-Oriented Methods: Principles & Practice. Addison-Wesley, 2000. 832 pages.

C. Szyperski, Component Software: Beyond Object-Oriented Programming. Addison Wesley, 2002.


R. T. Fielding, Architectural Styles and the Design of Network-based Software Architectures. PhD

thesis, University of California, Irvine, 2000. 162 pages.

P. Dix, Service-Oriented Design with Ruby and Rails. Addison-Wesley Professional, 2010. 320


N. Marz and J. Warren, Big Data: Principles and best practices of scalable realtime data systems.

Manning Publications, 2015. 328 pages.

T. Erl, C. Gee, P. Chelliah, J. Kress, H. Normann, B. Maier, L. Shuster, B. Trops, T. Winterberg,

C. Utschig, and P. Wik, Next Generation SOA: A Concise Introduction to Service Technology &

Service-Orientation. Pearson Education, 2014. 185 pages.

A. Dey, G. Chinchwadkar, A. Fekete, and K. Ramachandran, Metadata-as-a-service," in Inter-

national Conference on Data Engineering Workshops, pp. 6{9, IEEE, April 2015.

N. Viennot, M. Lecuyer, J. Bell, R. Geambasu, and J. Nieh, Synapse: A microservices architecture

for heterogeneous-database web applications," in European Conference on Computer Systems,

pp. 1{16, April 2015.

M. Villamizar, O. Garces, H. Castro, M. Verano, L. Salamanca, R. Casallas, and S. Gil, Evaluating

the monolithic and the microservice architecture pattern to deploy web applications in the cloud,"

in Computing Colombian Conference, pp. 583{590, September 2015.

K. Park, M. C. Nguyen, and H. Won, Web-based collaborative big data analytics on big data as a

service platform," in International Conference on Advanced Communication Technology, pp. 564{

, IEEE, July 2015.

V. D. Le, M. M. Ne , R. V. Stewart, R. Kelley, E. Fritzinger, S. M. Dascalu, and F. C. Har-

ris, Microservice-based architecture for the NRDC," in International Conference on Industrial

Informatics, pp. 1659{1664, July 2015.

M. Barrenechea, K. M. Anderson, L. Palen, and J. White, Engineering crowdwork for disaster

events: The human-centered development of a lost-and-found tasking environment," in Hawaii

International Conference on System Sciences, pp. 182{191, IEEE, 2015.

A. Popoola, D. Krasnoshtan, A.-P. Toth, V. Naroditskiy, C. Castillo, P. Meier, and I. Rahwan,

Information veri cation during natural disasters," in International Conference on the World

Wide Web, (New York, NY, USA), pp. 1029{1032, ACM, April 2013.

M. A. Cameron, R. Power, B. Robinson, and J. Yin, Emergency situation awareness from twitter

for crisis management," in 21st International Conference Companion on the World Wide Web,

pp. 695{698, ACM, April 2012.

L. Han, S. Potter, G. Beckett, G. Pringle, S.Welch, S.-H. Koo, G.Wickler, A. Usmani, J. L. Torero,

and A. Tate, Firegrid: An e-infrastructure for next-generation emergency response support,"

Journal of Parallel and Distributed Computing, vol. 70, no. 11, pp. 1128{1141, 2010.

P. Tin, T. T. Zin, T. Toriu, and H. Hama, An integrated framework for disaster event analysis

in big data environments," in International Conference on Intelligent Information Hiding and

Multimedia Signal Processing, pp. 255{258, IEEE, October 2013.

J. Heer and S. Kandel, Interactive analysis of big data," Crossroads: The ACM Magazine for

Students, vol. 19, pp. 50{54, September 2012.

S. Kandel, A. Paepcke, J. Hellerstein, and J. Heer, Wrangler: Interactive visual speci cation of

data transformation scripts," in Human Factors in Computing Systems, pp. 3363{3372, May 2011.

K. Wongsuphasawat, J. A. G. Gomez, C. Plaisant, T. D. Wang, M. Taieb-Maimon, and B. Shnei-

derman, Life

ow: Visualizing an overview of event sequences," in Human Factors in Computing

Systems, pp. 1747{1756, May 2011.

M. Oussalah, F. Bhat, K. Challis, and T. Schnier, A software architecture for twitter collection,search, and geolocation services," Knowledge-Based Systems, vol. 37, pp. 105{120, 2013.

M. A. Saleem, B. Varghese, and A. Barker, Bigexcel: A web-based framework for exploring big

data in social sciences," in International Conference on Big Data, pp. 84{91, IEEE, October 2014.

S. Verma, W. Corvey, S. Vieweg, J. Martin, L. Palen, M. Palmer, A. Schram, and K. M. Ander-

son, NLP to the rescue?: Extracting `situational awareness' tweets during mass emergency," in

International AAAI Conference on Weblogs and Social Media, pp. 385{392, July 2011.