Soft Theory
As researchers, we are investigating a multitude of phenomena in a deductive (i.e., quantitative) fashion. One typical problem we are facing is the type of data analysis best suited for our research problem. Controlled experiments, for example, might lead to insights into the specific mechanisms underpinning a certain practice, such as distributed development, pair programming, and test-driven development. However, at an initial stage of research, such highly controlled studies may not be feasible. In other domains, it may not be clear what the key constructs are, so that effective measurement cannot be done. Instead, researchers might opt for pragmatic alternative research approaches that do not require experimental control or active intervention in a study’s setting.
In this paper, we advocate the use of soft theory (based on soft modelling techniques) for quantitative studies in software engineering research, such as Partial Least Squares – Structural Equation Modelling.
Structural Equation Modeling
Structural Equation Modeling (SEM) is a powerful statistical tool to analyze complex relationships between latent variables (e.g., computational thinking, software quality, happiness, performance, productivity) if you need a more nuanced understanding compared to, e.g., a regression, or are analyzing longitudinal data.
In a paper published in ACM Computing Surveys, we take a closer look into the use of Partial Least Squares – Structural Equation Modeling (PLS-SEM) in Software Engineering Research. In that paper, we outline an updated methodological guideline for both authors and reviewers dealing with PLS-SEM.
Empirical Standards
Like any human activity, also peer-review (i.e., the revision process that a manuscript has to undergo before publication) is flawed and biased. An interesting experiment showed that 89% of reviewers would recommend rejecting a paper that has already been published in that very same journal (Peters & Ceci, 2010). Recently, in the field of computing, we started a discussion about how to improve our publication system (Vardi, 2021).
In Software Engineering, we came up with the idea of Empirical Standards, where I led the development of the longitudinal standard. The idea is simple and powerful: authors and reviewers will be guided by a pre-defined list of essential, desirable, and extraordinary elements a paper must have before publication (tailored on the most appropriate method(s) used). In other words, it is a transparent decision tree. Authors know what is expected from them and reviewers save time is assessing the relevant elements of the standard. Through this automatic checklist, the process becomes even easier.
