SERE: Software Engineering for the Research Environment

int i, j, k;
while… Viva presentation
Andre Oboler Lancaster
University
Computing Department
Lancaster University

8th October 2007

272+212

Thesis focus
• This thesis presents a new process based approach to
software engineering designed to meet the needs of
academic Computer Science researchers.

• The core objective was to examine whether software

engineering approaches could be adapted for the
research environment so that they gain acceptance
and enable improvement of the research process.
272+212

Limitations in scope
Academic research environment:
• While the approach and tools may have practical application in an
industrial R&D environment, this is outside the scope of this work

Computer Science Research:

• While some process descriptors are generic in nature, the focus of this work
is for research that involves the development of experimental software

• We assume a graduate level of computer science background amongst the

researchers, our approach seems to build on this and focus it on the needs
of research
272+212

Hypothesis
• A RAISER-based approach (building on our
past work) can improve the research process in
computer science academia.

• The null hypothesis is that a new approach will

make no difference.
272+212

Outcomes
• The data from this thesis leads us to reject the
null hypothesis on the basis that:
– Empirical data shows an improved outcome
– Qualitative feedback showed a positive experience
– Process descriptors demonstrated benefits
– The process perspective aids researchers in
reflection and enables further improvements
272+212

Contributions from this work

• The Process Model for Supporting Research
• The Personal Process Model
• Formal Technical Reviews for research
• Documentation standard for research
• The Research Process Optimisation Framework
• The high level framework for CARE
 First International Workshop on Improving Research
Productivity
272+212

PhD Process Review

• This research was undertaken in an evolutionary manner.
• Though necessary to development the SERE framework, the
final cycle (and post experiment analysis) leaves SERE much
more mature than it was in the early cycles.
• Though unavoidable, three years of data on the process as it
stands now would be an improvement over the data we have
• With plenty of scope for further development, even given a
further three years the approach would continue to evolve
• Inclusion of PhD candidates in the participant sample would
be another improvement, but was not possible in the scope of
this research
• The cyclic nature of the work led to some difficulties in thesis
structure, however the current mixed structure resolves these
nicely (covering both the cycles and the big picture)
272+212

Experimental difficulties
• By necessity participation was voluntary
+ve gives a true picture on adoptability
-ve smaller than idea sample sizes
• Need to provide full support myself
+ve first hand experience with all the data
-ve limiting factor on sample size and what could be done
• Timing not always ideal but dictated by circumstances
• Interview data not full processed
– No budget for transcription & very time consuming
– Some interviews pre-selected then transcribed (other not transcribed)
• Getting survey responses from students was a problem
• Limiting the thesis scope has been difficult as the work
touched upon many interesting and related topics.
272+212

Future developments
• Expansion of provision to PhD students
• Examination of the impact of a full Software
Development Lab
• Examination of specialised group templates for
Personal Process Models
• Examination of effectiveness and adaptation to
industrial Research & Development
• Further development of CARE
• Create of new and more specialised process
descriptors for different research areas
272+212

Viva discussion
There are further slides if you wish on:
• Motivation and reply to detractors
• The nature and needs of research
• The research process optimisation framework
• Improving knowledge transfer
• The Personal Process Model
• This research’s Personal Process Model
272+212

Motivation and
reply to retractors
272+212

Objections
One may ask, why bother?
• Current approaches have been working for
decades
• This seems like a lot of extra work
• Experienced researchers KNOW what they are
doing
• The struggle and reinvention of ideas is part of
the learning process
272+212

Response
• Increased pressure on science to deliver
• Increased pressure on students to complete faster
• Increased competition for resources
• Computer science is not yet mature
• Information Technology is now a part of life,
academia needs to be more scientific to hold value
• We can and should do better.
272+212

Nature and needs of research

272+212

Research is…

• “original investigation undertaken in order to gain

knowledge and understanding”[1]

• “creative work undertaken on a systematic basis in

order to increase the stock of knowledge” [2]
272+212

Systematic yet Creative?

• Reducing wasted effort is a type of improvement

that does not impact on creativity

• Not all effort is “wasted effort” so not all automation

is “safe” – some might reduce scientific creativity

• Remember: research is creative work undertaken on

a systematic basis, creativity and a systematic
nature must coexist
272+212

The Research Process

Optimisation Framework
272+212 The Research Process
Optimisation Framework
Effort is the volume filling the pipe per Standard Knowledge Unit

a b c d e f

Knowledge (SKU)

Reflects the fact that some parts of the research

process take more effort than others, and the amount
of this that is “wasted” also varies.
272+212
…as a bar chart
8

4 SKU
SEU

0
a b c d e f

Our aim should be to improve the Effort : Knowledge ratio,

either by reducing (wasted) effort or by increasing the
amount of knowledge that can be gained from that effort.
272+212

Improving knowledge transfer

272+212
Improving knowledge transfer
• How do we improve knowledge transfer?
– Capture process information
– Share it
– Develop it systematically
– Allow feedback

• We must also ensure that:

– Creativity is not inhibited
– “Wasted effort” is not increased

(overly bureaucratic solutions provide one counter examples that both

adds waste and inhibits creativity)
272+212
Approaches to
knowledge transfer
Artifacts:
• Research journal
• Electronic research journal (so searchable)
• Coding Guidelines
• Personal Process Model (PPM)

Process:
• Technical Reviews
272+212

Personal Process Model

272+212

Personal Process Model

• Uses a a java source code file to set out the research process
as a set of classes
• Documents the key parts of the research process using
comments
• Encapsulate and relate ideas using class relationships
• Generate external documentation, class diagrams and
hypermedia using dOxygen

• Monitor changes in ideas / plans using diff

• Discuss changes with supervisor and
colleagues, gain their wisdom MY
STUFF
272+212

Key advantages - researcher

• No context switching between coding environment and
research management environment
• Builds on exiting software engineering knowledge
• Uses an existing shared language for representation
• Diagrams are easy to read and follow, but don’t need to be
created / updated by hand
• Html output is easy to navigate and easy to share
• Process Comments can link to internal code documentation

MY
STUFF
272+212

Key advantages - research

For research as a whole, this approach means:

• More data is captured about the particular research

• More data is shared about research approaches
• Researchers share best practise and improve upon it

YOUR MY
STUFF STUFF
272+212

This research’s
Personal Process Model
272+212

My research model
272+212

My research model
Next slide will
expand this part
272+212

Zooming in…
We are discussing this
(please forgive the recursion)
272+212

References
[1] "RAE Circular 2/00" vol. 2006: Higher
Education Research Organisation, 2000

[2] OECD, The Measurement of Scientific and

Technological Activities - Frascati Manual 2002 :
Proposed Standard Practice for Surveys on
Research and Experimental Development Frascati,
France: Organisation for Economic Co-operation
and Development, 2002.