We have come across references stating that if “the standard software development lifecycle can be applied, an experiment is not required”. We believe this is a fundamentally flawed statement. Here is why:

The standard software development lifecycle involves six phases, namely:

  1. Requirement gathering and analysis.

  2. Design.

  3. Implementation/coding.

  4. Testing.

  5. Deployment.

  6. Maintenance.

An experiment is taken to involve a systematic progression of work that proceeds:

  1. from hypothesis

  2. experiment

  3. observation and evaluation, and

  4. leads to logical conclusions

The above is also commonly referred to as a scientific method, which is commonly applied in a range of industries.

Experimentation is the foundation of the scientific method, which is a systematic means of testing if something works or is true. Although some experiments take place in laboratories, it is possible to perform an experiment anywhere, at any time, even in software development.

Following a process, does not preclude an experiment having occurred, or being undertaken. In fact it reinforces a a systematic progression of work has been followed; not ad-hoc, unstructured ‘trial and error’. The standard software development lifecycle is a process, which enables experiments to be undertaken.

  • Whether or not an experiment has occurred is predicated on having a hypothesis which is:

“a particular technical or scientific idea and is commonly expressed as a relationship between variables (or technical features) which can be proven or disproven.”

The hypothesis is the idea being investigated through the systematic progression of work. The hypothesis will generally direct the design and conduct of the experiment, observation and evaluation.

Applying this to the standard software development lifecycle, the hypothesis typically occurs during phases 1. and 2 (which involve research and design).

A hypothesis in software development involves…

  • an experiment is defined by the Cambridge Dictionary as:

“a test done in order to learn something or to discover if something works or is true.”

Applying this to the standard software development lifecycle, testing typically occurs during phases 3., 4. and can even occur in 5 (which involve prototypes at various levels, and testing of these).

  • observation and evaluation is the phase where the results of the experiment/testing are assessed against the hypothesis to determine if it has been proven/disproven.

Applying this to the standard software development lifecycle, results of testing are analysed to assess if the accuracy and performance metrics associated with a hypothesis have been met. Again, this typically occurs during phases 3., 4. and can even occur in 5 of the standard software development lifecycle (which involve prototypes at various levels, and testing of these).