A Summary of the Workshop Position Papers

The modern profession of engineering was born almost hundred years ago when exploding boilers and collapsing iron bridges threatened the new found prosperity of the industrial revolution. The lessons learned from these tragedies evolve engineering from a craft with the knowledge of masters passed parent to child into to a profession with an established body of knowledge.   Engineering is often referred to as  “applied science” where scientific principles from physics, chemistry, and math are applied to design problems. Engineers often boast about the precision required by our discipline, often by citing wonderful engineering trivia such as how even a 1 part in 100,000 error would have lead to disaster in the Apollo 11 mission, or how due to the Earth’s curvature, the towers of a large suspension bridges are 2 cms further apart at the top than at the base.   These boasts demonstrate not only the pride engineers take in their profession but underscore the duty because it is this relentless attention to detail that makes engineer structures safe and reliable. Through education and culture, engineers are socialized their duty requires us not to tolerate ambiguity and fuzzy thinking. It is often rumoured (falsely) the iron rings sported by Canadian engineers are made from the wreckage of 1907 Quebec bridge collapse as a reminder of that duty.

However some of the new challenges facing our profession cannot be resolved with this strict positivist approach.  While positivist thinking provides us with a good handle on problems engineering is intended to solve, one of most significant challenges facing our profession is the effective organization of people. This is especially true for software because unlike other engineered artifact there are few scientific principles to constrain software and therefore almost anything we can imagine we can make happen. Software development is primarily a social process. Barry Boehme, one of software engineering’s great patrician wrote  “Personnel attributes and human relations activities provide by far the largest source of opportunity for improving software productivity”(Boehm, 1984)

Our position is social processes are too complex to understand using a strict positivist approach. The issues are too rich and too complex for generalizable results to have any utility.  This means we must venture into an area many engineers suspect as fuzzy and ambiguous – if you can’t put a number to Glass’ analysis of software engineering research which revealed less than 1% of research papers explored organizational issues or employed research methods useful in understanding social behaviour. (Glass et al., 2002)

Some may see our pursuit of qualitative methods ins software engineering as a dilution or even corruption of the positivist approach that made engineering the profession it is today.  Others may suggest that social research is best left to sociologists. We believe this is an opportunity for the profession to grow and that no one else is in a better position to understand our problems than ourselves.

However, few engineers are educated in the methods necessary for conducting social research.  We are engineers after all, we entered this field because we love the technical challenges engineering put before us. But for our discipline to grow, we have to grow.  Few of us know how to judge qualitative research, how to distinguish opinion offered as research versus interpretation. If we cannot judge and if the quality of our research is spotty, then how can we expect our colleagues to accept and trust our results? In our own review of qualitative research in software engineering we found even award winning papers wanting. Engineers love rules and standards, and in our opinion, we need to establish a standard for what constitutes high quality qualitative research, not only for researchers, but also for reviewers, publishers and the engineering audience in general. In short, engineers need to know they can trust the results.

This workshop started as a proposal to the 2009 International Conference on Software Engineering (ICSE) and after it was declined by the ICSE we decided to continue with it anyways. This topic is too important to our community to simply wait to try again next time. Many of the interested parties would be in Vancouver for ICSE and this was not an opportunity to be missed. We let the ICSE organizers know what we were up to and organized the workshop.  In January 2009 we created a blog requesting position papers and received 7 position papers. One of the luxuries of running a small unaffiliated workshop is we can be flexible in our organization of the workshop. We have continued to receive submission and now have 10 position papers. Some are brief positions (Seaman, Carver, Adolph) and  others are experience reports, and finally some are analysis of research.

A theme that ran through many of the papers is software engineering is different from traditional engineering disciplines and qualitative research offers a better fit for understanding software engineering problems. Carver, Dittrich, Mannisto and Raatikainen offered papers arguing why qualitative research is important to understanding software engineering. Seaman and Adolph advanced positions for how to grow and advance qualitative research in software engineering.  Sillito and Alwis described tools that may improve qualitative methods.

Finally, Philippe and I want to extend our sincer thanks to all who participant in this workshop. Our fondest hope is we are able to knit together a disparate community to develop a standard for qualitative research in software engineering that can be published as a “Handbook of Qualitative Research in Software Engineering”

Yvonne Dittrich – Understanding Software Engineering as Situated Practice

Yvonne Dittrich expresses a view shared by many that software engineering is the artful integration of technologies, tools and methods into situated practices of software development. Context matters. Unfortunately the pursuit to discover generalizable results usually requires we abstract that context away.  

Jeffery Carver

Jeffery Carver emphasizes the importance of social relationships in software engineering with the position that software engineering research is software engineers performing behavioural research on software engineers. Dr Carver describes two studies and their protocols and interest in this workshop is to share and exchange ideas and to learn from others.

Tomi Männistö and Mikko Raatikainen Qualitative research methods in Software Engineering at SoberIT— A position paper

Mannisto and Raatikainen take the position we need to acknowledge the multi-paradigmatic and multi-methodological nature of software engineering research that covers both technical and social aspects. qualitative research methods provide a better fit for the problems they are addressing.

Carolyn Seaman – The Potential Role of ISERN in the promotion of qualitative research in software engineering

One of the goals of this workshop is to look for ways to broaden the qualitative research community by knitting together those who are actively involved or interested in qualitative methods. Carolyn Seaman advocates using ISERN to further the use of qualitative methods in software engineering.

Steve Adolph – Reaching Out

Steve Adolph takes the position the quality of software engineering qualitative research is uneven at best.  Engineers do not have the education and training as sociologists and our discipline does not have the experience to create an effective qualitative research agenda. We need to reach out to the other disciplines that have expertise and experience conducting high quality qualitative research.

Jonathan Sillito and Brian de Alwis:  Saturate: A collaborative memoing tool

Research is tedious at best, and qualitative research sometimes takes tedium to the next level -  just ask researcher doing grounded theory. In engineering, tedium often translates to opportunity, opportunity to find a tool to help reduce the tedium. Sillito and Alwis describe a tool which may help trawl the vast amount of data available in public archives, issue tracking systems, discussion groups, document. Can tools make a difference?  

Rafael Prikladnicki and Alexander Boden: Towards an Understanding of the Different Dimensions of Conducting Qualitative Research in Global Software Engineering

The world is flat and Prklandnicki and Boden present a model that illustrate different dimensions and challenges for conducting qualitative research in Global Software Enginering, concentrating on data collection strategies, specifically, the location of the object under study and the location of the research team. What influence does this have on study design and budget? How does this limit the degrees of freedom a research may have in their choice of methods? (for example, cost of travel may reduce the option for longitudinal studies when many sites are involved).

P&B identify and open an important issue when it comes to funding for GSE – selecting a research partner. What are the issues when conducting  qualitative research as part of a larger study?   

Maryam Najafian Razavi - A Grounded Theory of Information Sharing Behavior in Social Software – Challenges and an Experience

Maryam Razavi offers a glimpse of a qualitative study using grounded theory when she investigates the issues of privacy management in social networking.