GETTING THE QUERY RIGHT FOR CRISIS INFORMATICS DESIGN ISSUES FOR WEB-BASED ANALYSIS ENVIRONMENTS
Keywords:
data-intensive systems, web applications, design issues, crisis informaticsAbstract
Web-based data analysis environments are powerful platforms for exploring large data sets. To ensure that these environments meet the needs of analysts, a human-centered design perspective is needed. Interfaces to these platforms should provide exible search, support user-generated content, and enable collaboration. We report on our eorts to design and develop a web interface for a custom analytics platform|EPIC Analyze| which provides interactive search over large Twitter data sets collected during crisis events. We performed seven think-aloud sessions with researchers who regularly analyze crisis data sets and compiled their feedback. They identied a need for a \big picture" view of an event, exible querying capabilities, and user-dened coding schemes. Adding these features allowed EPIC Analyze to meet the needs of these analysts and enable exploratory research on crisis data. In performing this work, we identied an opportunity to migrate the software architecture of EPIC Analyze to one based on microservices. We report on the lessons learned in performing this migration and the impact it had on EPIC Analyze's capabilities. We also re ect on the benets a microservices approach can have on the design of data-intensive software systems like EPIC Analyze.
Downloads
References
K. M. Anderson, Embrace the challenges: Software engineering in a big data world," in First
International Workshop on Big Data Software Engineering, Part of the 2015 International Con-
ference on Software Engineering, pp. 19{25, May 2015.
L. Palen, K. M. Anderson, G. Mark, J. Martin, D. Sicker, M. Palmer, and D. Grunwald, A
vision for technology-mediated support for public participation & assistance in mass emergencies
& disasters," in ACM-BCS Visions of Computer Science, April 2010. Article 8. 12 pages.
K. M. Anderson and A. Schram, Design and implementation of a data analytics infrastructure
in support of crisis informatics research (nier track)," in International Conference on Software
Engineering, pp. 844{847, May 2011.
K. M. Anderson, A. Schram, A. Alzabarah, and L. Palen, Architectural implications of social me-
dia analytics in support of crisis informatics research," IEEE Bulletin of the Technical Committee
on Data Engineering, vol. 36, pp. 13{20, September 2013.
A. Schram and K. M. Anderson, MySQL to NoSQL: Data modeling challenges in supporting scal-
ability," in Systems, Programming Languages, and Applications: Software for Humanity, pp. 191{
, October 2012.
K. M. Anderson, A. A. Aydin, M. Barrenechea, A. Cardenas, M. Hakeem, and S. Jambi, Design
challenges/solutions for environments supporting the analysis of social media data in crisis in-
formatics research," in Hawaii International Conference on System Sciences, pp. 163{172, IEEE,
January 2015.
D. Fisher, R. DeLine, M. Czerwinski, and S. Drucker, Interactions with big data analytics,"
Interactions, vol. 19, pp. 50{59, May + June 2012.
L. Palen, S. Vieweg, J. Sutton, S. Liu, and A. Hughes, Crisis informatics: Studying crisis in a
networked world," in Third International Conference on E-Social Science, October 2007. 10 pages.
A. Sarcevic, L. Palen, J. White, K. Starbird, M. Bagdouri, and K. M. Anderson, `Beacons of
hope' in decentralized coordination: Learning from on-the-ground medical twitterers during the
haiti earthquake," in Computer Supported Cooperative Work, pp. 47{56, February 2012.
A. L. Hughes, L. St. Denis, L. Palen, and K. M. Anderson, Online public communications by
police & re services during the 2012 hurricane sandy," in Human Factors in Computing Systems,
pp. 1505{1514, April 2014.
L. Huang, K. Starbird, M. Orand, S. Stanek, and H. Pedersen, Connected through crisis: Emo-
tional proximity and the spread of misinformation online," in Computer Supported Cooperative
Work, pp. 969{980, March 2015.
M. Barrenechea, K. M. Anderson, A. A. Aydin, M. Hakeem, and S. Jambi, Getting the query
right: User interface design of analysis platforms for crisis research," in International Conference
on Web Engineering, pp. 547{564, June 2015.
C. Hagar, Crisis informatics: Perspectives of trust|is social media a mixed blessing?," iSchool
Student Research Journal, vol. 2, no. 2, 2013.
K. Starbird and L. Palen, `Voluntweeters': Self-organizing by digital volunteers in times of crisis,"
in Human Factors in Computing Systems, pp. 1071{1080, May 2011.
J. White, L. Palen, and K. M. Anderson, Digital mobilization in disaster response: The work &
self-organization of on-line pet advocates in response to hurricane sandy," in Computer Supported
Cooperative Work and Social Computing, pp. 866{876, February 2014.
K. Starbird, L. Palen, A. Hughes, and S. Vieweg, Chatter on the red: What hazards threat
reveals about the social life of microblogged information," in Computer Supported Cooperative
Work, pp. 241{250, February 2010.
L. Palen, R. Soden, T. J. Anderson, and M. Barrenechea, Success and scale in a data-producing
organization: The socio-technical evolution of openstreetmap in response to humanitarian events,"
in Human Factors in Computing Systems, pp. 4113{4122, April 2015.
J. Bargas-Avila and K. Hornbk, Foci and blind spots in user experience research," Interactions,
vol. 19, pp. 24{25, November + December 2012.
C. McTaggart, Analysis and implementation of software tools to support research in crisis infor-
matics," Master's thesis, University of Colorado, 2012. 65 pages.
J. Laconich, J. Jose, F. Casati, and M. Marchese, Social spreadsheet," in Web Engineering
(F. Daniel, P. Dolog, and Q. Li, eds.), vol. 7977 of Lecture Notes in Computer Science, pp. 156{170,
Springer Berlin Heidelberg, 2013.
C. Lewis and J. Rieman, Task-centered User Interface Design: A Practical Introduction. Depart-
ment of Computer Science, University of Colorado, Boulder, 1993. 170 pages.
B. Shneiderman, The eyes have it: A task by data type taxonomy for information visualizations,"
in IEEE Symposium on Visual Languages, pp. 336{343, September 1996.
A. A. Aydin and K. M. Anderson, Incremental sorting for large dynamic data sets," in First
IEEE International Conference on Big Data Computing Service and Applications, pp. 170{175,
March + April 2015.
C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins, Pig latin: A not-so-foreign lan-
guage for data processing," in International Conference on the Management of Data, SIGMOD
'08, (New York, NY, USA), pp. 1099{1110, ACM, June 2008.
M. Fowler and J. Lewis, Microservices," March 2014. http://martinfowler.com/articles/
microservices.html.
I. Graham, Object-Oriented Methods: Principles & Practice. Addison-Wesley, 2000. 832 pages.
C. Szyperski, Component Software: Beyond Object-Oriented Programming. Addison Wesley, 2002.
pages.
R. T. Fielding, Architectural Styles and the Design of Network-based Software Architectures. PhD
thesis, University of California, Irvine, 2000. 162 pages.
P. Dix, Service-Oriented Design with Ruby and Rails. Addison-Wesley Professional, 2010. 320
pages.
N. Marz and J. Warren, Big Data: Principles and best practices of scalable realtime data systems.
Manning Publications, 2015. 328 pages.
T. Erl, C. Gee, P. Chelliah, J. Kress, H. Normann, B. Maier, L. Shuster, B. Trops, T. Winterberg,
C. Utschig, and P. Wik, Next Generation SOA: A Concise Introduction to Service Technology &
Service-Orientation. Pearson Education, 2014. 185 pages.
A. Dey, G. Chinchwadkar, A. Fekete, and K. Ramachandran, Metadata-as-a-service," in Inter-
national Conference on Data Engineering Workshops, pp. 6{9, IEEE, April 2015.
N. Viennot, M. Lecuyer, J. Bell, R. Geambasu, and J. Nieh, Synapse: A microservices architecture
for heterogeneous-database web applications," in European Conference on Computer Systems,
pp. 1{16, April 2015.
M. Villamizar, O. Garces, H. Castro, M. Verano, L. Salamanca, R. Casallas, and S. Gil, Evaluating
the monolithic and the microservice architecture pattern to deploy web applications in the cloud,"
in Computing Colombian Conference, pp. 583{590, September 2015.
K. Park, M. C. Nguyen, and H. Won, Web-based collaborative big data analytics on big data as a
service platform," in International Conference on Advanced Communication Technology, pp. 564{
, IEEE, July 2015.
V. D. Le, M. M. Ne , R. V. Stewart, R. Kelley, E. Fritzinger, S. M. Dascalu, and F. C. Har-
ris, Microservice-based architecture for the NRDC," in International Conference on Industrial
Informatics, pp. 1659{1664, July 2015.
M. Barrenechea, K. M. Anderson, L. Palen, and J. White, Engineering crowdwork for disaster
events: The human-centered development of a lost-and-found tasking environment," in Hawaii
International Conference on System Sciences, pp. 182{191, IEEE, 2015.
A. Popoola, D. Krasnoshtan, A.-P. Toth, V. Naroditskiy, C. Castillo, P. Meier, and I. Rahwan,
Information veri cation during natural disasters," in International Conference on the World
Wide Web, (New York, NY, USA), pp. 1029{1032, ACM, April 2013.
M. A. Cameron, R. Power, B. Robinson, and J. Yin, Emergency situation awareness from twitter
for crisis management," in 21st International Conference Companion on the World Wide Web,
pp. 695{698, ACM, April 2012.
L. Han, S. Potter, G. Beckett, G. Pringle, S.Welch, S.-H. Koo, G.Wickler, A. Usmani, J. L. Torero,
and A. Tate, Firegrid: An e-infrastructure for next-generation emergency response support,"
Journal of Parallel and Distributed Computing, vol. 70, no. 11, pp. 1128{1141, 2010.
P. Tin, T. T. Zin, T. Toriu, and H. Hama, An integrated framework for disaster event analysis
in big data environments," in International Conference on Intelligent Information Hiding and
Multimedia Signal Processing, pp. 255{258, IEEE, October 2013.
J. Heer and S. Kandel, Interactive analysis of big data," Crossroads: The ACM Magazine for
Students, vol. 19, pp. 50{54, September 2012.
S. Kandel, A. Paepcke, J. Hellerstein, and J. Heer, Wrangler: Interactive visual speci cation of
data transformation scripts," in Human Factors in Computing Systems, pp. 3363{3372, May 2011.
K. Wongsuphasawat, J. A. G. Gomez, C. Plaisant, T. D. Wang, M. Taieb-Maimon, and B. Shnei-
derman, Life
ow: Visualizing an overview of event sequences," in Human Factors in Computing
Systems, pp. 1747{1756, May 2011.
M. Oussalah, F. Bhat, K. Challis, and T. Schnier, A software architecture for twitter collection,search, and geolocation services," Knowledge-Based Systems, vol. 37, pp. 105{120, 2013.
M. A. Saleem, B. Varghese, and A. Barker, Bigexcel: A web-based framework for exploring big
data in social sciences," in International Conference on Big Data, pp. 84{91, IEEE, October 2014.
S. Verma, W. Corvey, S. Vieweg, J. Martin, L. Palen, M. Palmer, A. Schram, and K. M. Ander-
son, NLP to the rescue?: Extracting `situational awareness' tweets during mass emergency," in
International AAAI Conference on Weblogs and Social Media, pp. 385{392, July 2011.
