Software Construction & Design 1
The University of Sydney Page 1
Agile Software
Development Practices
SOF2412 / COMP9412
System Build Automation
Dr. Basem Suleiman
School of Information Technologies
The University of Sydney Page 2
Agenda
– Software Configuration Management
– System Building
– Agile System Build
– Software Build Automation Tools
– Ant
– Maven
– Gradle
The University of Sydney Page 3
Software Configuration
Management
The University of Sydney Page 4
Revisit – Software Development Process Model
Requirements
Design
Implementation &
Test & Integration
& More Design
Final Integration
& System Test
Requirements
Design
3 weeks (for example)
The system grows
incrementally.
Feedback from
iteration N leads to
refinement and
adaptation of the
requirements and
design in iteration
N+1.
Iterations are fixed in
length, or timeboxed.
Time
Implementation &
Test & Integration
& More Design
Final Integration
& System Test
• Rational Unified Process (RUP) iterative and incremental approach to develop OO
software systems
The University of Sydney Page 5
Agile Development – Increments
Work for future
iterations
Work for the
current
iteration
Every 4 weeks, produce a
new shippable product
• Software is incrementally developed
Agile Vs Waterfall: Showdown For Software Development Domination
Agile Vs Waterfall: Showdown For Software Development Domination
The University of Sydney Page 6
Configuration Management (CM)
– Configuration management (CM) is concerned with the policies, processes
and tools for managing changing software systems
– Track of what changes and component versions incorporated into each
system version
– Essential for team projects to control changes made by different developers
The University of Sydney Page 7
Configuration Management Activities
The University of Sydney Page 8
Configuration Management Activities
– System building: assembling program components, data and libraries, then
compiling these to create an executable system
– Version management: keeping track of the multiple versions of system
components and ensuring that changes made to components by different
developers do not interfere with each other
– Change management: keeping track of requests for changes to the
software from customers and developers, working out the costs and impact
of changes, and deciding the changes should be implemented
– Release management: preparing software for external release and
keeping track of the system versions that have been released for customers
The University of Sydney Page 9
Multi-version System Development
Ian Sommerville. 2016. Software Engineering (10th ed.)
The University of Sydney Page 11
Version Management (VM)
– Keep track of different versions of software components or configuration
items and the systems in which these components are used
– Ensuring changes made by different developers to these versions do not
interfere with each other
– The process of managing code-lines and baselines
The University of Sydney Page 12
Baselines
Ian Sommerville. 2016. Software Engineering (10th ed.)
The University of Sydney Page 13
Version Management – Codelines and Baselines
– Version Management can be thought of as the process of managing codelines
and baselines
– Codeline: a sequence of versions of source code with later versions in the
sequence derived from earlier versions
– System’s components often have different versions
– Baseline: a definition of a specific system
– Specifies the component versions that are included in the system plus a
specification of the libraries used, configuration files, etc.
The University of Sydney Page 14
Baselines
– Baselines may be specified using a configuration language, to define what
components are included in a version of a particular system
– Useful for recreating a specific version of a complete system
– E.g., individual system versions for different customers. If a customer reports bugs
in their system one can recreate the version delivered to a specific customer
The University of Sydney Page 16
Semantic Versioning (SemVer)
– Set of rules and requirements that determine how version numbers should be assigned
and incremented for software being developed
– Semantic numbers; numbers with meaning in relation to a certain version
– Why?
– Managing versioning numbers in a meaningful and standard way
– Managing dependencies: the bigger your system grows, the more
packages/libraries/plugins you integrate into your software
– Given a version number MAJOR.MINOR.PATCH, increment the:
1. MAJOR version when you make incompatible API changes,
2. MINOR version when you add functionality in a backwards-compatible manner,
3. PATCH version when you make backwards-compatible bug fixes.
https://semver.org/
https://semver.org/
The University of Sydney Page 17
Semantic Versioning – Example
https://docs.npmjs.com/getting-started/semantic-versioning
More details on semantic versioning – https://semver.org/
Stage Code Rule Example
New product 1st release Start with 1.0.0 1.0.0
Patch Release Bug fixes, other minor changes Increment the 3rd digit 1.0.1
Minor Release New features that do not break
existing features
Increment the 2nd digit 1.1.0
Major Release Changes that break backward
compatibility
Increment the 1st digit 2.0.0
https://docs.npmjs.com/getting-started/semantic-versioning
https://semver.org/
The University of Sydney Page 19
Agile Development in CM
– Agile development, where components and systems are changed several
times per day, is impossible without using CM tools
– The definitive versions of components are held in a shared project repository
and developers copy these into their own workspace
– They make changes to the code then use system building tools to create a new
system on their own computer for testing. Once they are happy with the
changes made, they return the modified components to the project repository.
The University of Sydney Page 20
System Building
The University of Sydney Page 21
System Building
– System building is the process of creating a complete, executable system by
compiling and linking the system components, external libraries,
configuration files, etc.
– System building tools and version management tools must communicate as
the build process involves checking out component versions from the repo
managed by the version management system.
– The configuration description used to identify a baseline is also used by the
system building tool
The University of Sydney Page 27
Software Build
Automation Tools
Ant, Maven, Gradle
The University of Sydney Page 28
Tools for Agile Development
Version
Control
Version
Control
Build
Automation
Build
Automation
The University of Sydney Page 29
Build Tools – Apache Ant
– Java-based software build tool for automating build processes
– Requires Java platform and best suited for building Java projects
– Does not impose coding conventions
– Does not impose any heavyweight dependency management framework
– XML to describe the code build process and its dependencies
– Default build.xml
The University of Sydney Page 30
Apache ANT – Example
https://en.wikipedia.org/wiki/Apache_Ant
https://en.wikipedia.org/wiki/Apache_Ant
The University of Sydney Page 31
Apache ANT – Drawbacks
– Too flexible
– Complexity (XML-based build files)
– Need to specify a lot of things to make simple builds
– No standard structure/layout
– Developers can create their own structure/layout of the project
The University of Sydney Page 32
Apache Maven
– A build automation tool primarily for java projects
– XML-based description of the software being built
– Dependencies on other external modules and components, the build order,
directories, and required plug-ins
– Central repository (e.g., Maven 2)
– Coding by convention: it uses conventions over configuration for the build
procedure
– Supported by Eclipse, IntelliJ, JBuilder, NetBeans
– Plugin-based architecture
– Plugin for the .NET framework and native plugins for C/C++ are maintained
The University of Sydney Page 33
Apache Maven – Minimal Example
– Maven projects are
configured using Project
Object Model (POM)
stored in a pom.xml file
The University of Sydney Page 34
Apache Maven – Project Structure
– The command mvn package will
– compile all the Java files
– run any tests
– package the deliverable code and resources into target
(e.g., target/my-app-1.0.jar)
https://commons.wikimedia.org/wiki/File:Maven_CoC.svg#/media/File:Maven_CoC.svg
The Maven software tool auto-generated this
directory structure for a Java project
https://commons.wikimedia.org/wiki/File:Maven_CoC.svg#/media/File:Maven_CoC.svg
The University of Sydney Page 35
Apache Maven – Central Repository
– Maven uses default Central Repository that maintains required software
artifacts (libraries, plug-ins) to manage dependencies
– E.g., project that is dependent on the Hibernate library needs to specify
that in the pom.xml project file
– Maven checks if the referenced dependency is already in the user’s local
repository
– It references the dependency from the local repository or
– Dynamically download the dependency and the dependencies that Hibernate itself
needs (transitive dependency) and store them in the user’s local repository
– You can configure repositories other than the default (e.g., company-
private repository)
The University of Sydney Page 36
Apache Maven – Drawbacks
– Again, XML-based files increase complexity (verbose)
– Rigid; developers are required to understand follow the conventions
The University of Sydney Page 37
Gradle
– Build automation tool that builds upon the concepts of Ant and Maven
– build conventions, and redefine conventions
– Project described using Groovy-based Domain Specific Language (DSL)
– Tasks orders determined using a directed acyclic graph (DAC)
– Multi-project builds
– Incremental builds; up to date parts of the build tree do not need to be re-
executed
– Dependency handling (transitive dependency management)
https://en.wikipedia.org/wiki/Gradle
https://en.wikipedia.org/wiki/Gradle
The University of Sydney Page 38
Gradle – Groovy
– Gradle build files are Groovy scripts
– Groovy is a dynamic language of the JVM
– Can be added as a plug-in
– Allows developers to write general programming tasks in the build files
– Relief developers from the lacking control flow in Ant or being forced into
plug-in development in Maven to declare nonstandard tasks
https://en.wikipedia.org/wiki/Gradle
https://en.wikipedia.org/wiki/Gradle
The University of Sydney Page 39
Gradle – DSL
– Gradle also presents a DSL tailored to the task of building code
– Not general-purpose or programming language
– Gradle DSL contains the language needed to describe how to build Java
code and create a WAR file from the output
– Gradle DSL is extensible through plug-ins
https://en.wikipedia.org/wiki/Gradle
https://en.wikipedia.org/wiki/Gradle
The University of Sydney Page 40
Gradle – Extensible DSL
– Extensibility using plug-ins (if it doesn’t have the language to
describe what you want your build to do)
– E.g., describe how to run database migration scripts or deploy code to a
set of cloud-based QA servers
– Gradle plug-ins allow:
– Adding new task definitions
– Change the behavior of existing tasks
– Add new objects
– Create keywords to describe tasks that depart from the standard Gradle
categories
The University of Sydney Page 41
Gradle Basics
The University of Sydney Page 42
Gradle – Tasks
– Task: a single atomic piece of work for a build
– e.g., compiling classes
– Project: a composition of several tasks
– e.g., Creation of a jar file
– Each task has a name, which can be used to refer to the task within its
owning project, and a fully qualified path, which is unique across all tasks in
all projects
https://en.wikipedia.org/wiki/Grale
https://en.wikipedia.org/wiki/Gradle
The University of Sydney Page 43
Gradle – Task Actions
– A task is made up of sequence of Action objects
– Action.execute(T) to execute a task
– Add actions to a task
– Task.doFirst() or Task.doLast()
– Task action exceptions
– StopActionException to abort execution of the action
– StopExecutionException to abort execution of the task and continue to the next
task
https://en.wikipedia.org/wiki/Grale
https://en.wikipedia.org/wiki/Grale
The University of Sydney Page 44
Gradle – Simplest Build File Example
$ gradle -q helloWorld
hello, world
Results of Hello World build file
Build.gradle
task helloWorld << { println 'hello, world' } Build.gradle task hello << { print 'hello, ' } task world(dependsOn: hello) << { println 'world' } execute the second task, world $ gradle -q world hello, world The University of Sydney Page 45 Gradle – Simplest Build File for Java Example (1) Project layout of HelloWorld.java Simplest possible Gradle file for java The University of Sydney Page 46 Gradle – Simplest Build File for Java Example (2) Project Layout of Hello World Java After Build gradle build Gradle automatically introduces number of tasks for us to run Note: • Class files generated and place in a directory • Test report files (for unit test results) • JAR built using the project directory Run HelloWorld Java $ java -cp build/classes/main/ org.gradle.example.simple.HelloWorld hello, world The University of Sydney Page 47 Gradle – Build Lifecycle – Phases of executing a build file in Gradle: – Initialization: projects are to participate in the build – Configuration: task objects are assembled into an internal object model called Directed Acyclic Graph (DAG) – Execution: build tasks are executed in the order required by their dependency relationship The University of Sydney Page 48 Gradle – Task Configuration – Configuration block: to setup variables and data structures needed by the task action when it runs in the build – Make tasks rich object model populated with information about the build – Runs during gradle’s configuration lifecycle when task actions executed – Closure: a block of code specified by curly braces – Holding blocks of configuration and build actions The University of Sydney Page 49 Gradle – Task Configuration Example Task actions Task Configuration block << to add action to a task (in Groovy) The University of Sydney Page 50 Gradle – Tasks are Objects – Every task is represented internally as an object – Task’s methods and properties – Gradle creates an internal object model of the build before executing it – Each new task is of DefaultTask type - task type can be changed – DefaultTask contains functionality required for them to interface with Gradle project model The University of Sydney Page 51 Gradle – Methods of Default Task Method Description dependsOn(task) Adds a task as a dependency of the calling task. A depended-on task will always run before the task that depends on it doFirst(closure) Adds a block of executable code to the beginning of a task’s action. During the execution phase, the action block of every relevant task is executed. doLast(closure) Appends behavior to the end of an action onlyIf(closure) Expresses a predicate which determines whether a task should be executed. The value of the predicate is the value returned by the closure. The University of Sydney Page 52 Gradle – dependsOn() Example Different ways to call the dependsOn method The University of Sydney Page 53 Gradle – doFirst() Example Call the doFirst on the task object (top) and inside task’s configuration block (bottom) Repeated calls to the doFirst method are additive. OR The University of Sydney Page 54 Gradle – onlyIf() Example Using onlyIf method to do simple system property tests – onlyIf method can be used to switch individual tasks on and off using any logic you can express in Groovy code – E.g., read files, call web services, check security credentials The University of Sydney Page 55 Gradle – Default Task’s Properties Method Description didWork A Boolean property indicating whether the task completed successfully enabled A Boolean property indicating whether the task will execute. path A string property containing the fully qualified path of a task (levels; DEBUG, INFO, LIFECYCLE, WARN, QUIET, ERROR) logger A reference to the internal Gradle logger object logging The logging property gives us access to the log level temporaryDir Returns a File object pointing to a temporary directory belonging to this build file. It is generally available to a task needing a temporary place in to store intermediate results of any work, or to stage files for processing inside the task description a small piece of human-readable metadata to document the purpose of a task The University of Sydney Page 56 Gradle – Dynamic Properties – Properties (other than built-in ones) can be assigned to a task – A task object functions can contain other arbitrary property names and values we want to assign to it (do not use built-in property names) The University of Sydney Page 57 Gradle Task Types – Copy – A copy task copies files from one place into another task copyFiles(type: Copy) { from 'resources' into 'target' include '**/*.xml', '**/*.txt', '**/*.properties' } Note: the from, into, and include methods are inherited from the Copy The University of Sydney Page 58 Gradle Task Types – Jar – A Jar task creates a jar file from source files – The Java plug-in’s jar creates a task of this type – It packages the main source set and resources together with a trivial manifest into a jar bearing the project’s name in the build/libs directory – highly customizable The University of Sydney Page 59 Gradle Task Types – Jar Example apply plugin: 'java' task customJar(type: Jar) { manifest { attributes firstKey: 'firstValue', secondKey: 'secondValue' } archiveName = 'hello.jar' destinationDir = file("${buildDir}/jars") from sourceSets.main.classes } The University of Sydney Page 60 Gradle Task Types – JavaExec – A JavaExec task runs a Java class with a main() method – Tries to take the hassle away and integrate command-line Java invocations into your build The University of Sydney Page 61 Gradle Task Types – JavaExec Example apply plugin: 'java' repositories { mavenCentral() } dependencies { runtime 'commons-codec:commons-codec:1.5' } task encode(type: JavaExec, dependsOn: classes) { main = 'org.gradle.example.commandline.MetaphoneEncoder' args = "The rain falls mainly in the plain".split().toList() classpath sourceSets.main.classesDir classpath configurations.runtime } The University of Sydney Page 62 Gradle Custom Task Types – Gradle’s built-in tasks might not be sufficient for all scenarios – Gradle allows defining custom task types in: – The Build File and must extend the DefaultTask class or one of its descendants – The Source Tree: when the task’s logic is significant has its own class hierarchy and might rely on external interface • When the custom task logic outgrows the build file, it can be migrated to the buildSrc directory at the project root • This directory is automatically compiled and added to the build classpath The University of Sydney Page 63 Gradle Custom Task Types – Build File – Suppose your build file needs to issue arbitrary queries against a MySQL database Task’s properties (Groovy idiom) * * * * Task method will run when the task runs * Actual build tasks inherits MySQLType’s properties and actions The University of Sydney Page 65 Gradle Custom Task Types – Source Tree – . Source tree provides a robust way for elaborating on that simple task behavior, growing an object model, writing tests, and doing everything else we normally do when developing software The University of Sydney Page 67 Gradle – Plug-ins Java Plug-in The University of Sydney Page 68 Gradle – Java Plug-in – A plug-in is an extension to Gradle which configures projects – Java plug-in adds some tasks to your project which will compile and unit test your Java source code, and bundle into a JAR – Convention-based; default values for many aspects of the project are pre- defined • In your build.gradle: apply plugin : ‘java’ – Can customize projects if you do not follow the convention The University of Sydney Page 69 Gradle – Java Plug-in (Project Structure) https://docs.gradle.org/current/userguide/java_plugin.html – Gradle expects to find production source code under src/main/java and test source code under src/test/java – Files under src/main/resources will be included in the JAR as resources – Files under src/test/resources will be included in the classpath used to run tests – All output files are created under the build directory, with the JAR file will end up in the build/libs directory https://docs.gradle.org/current/userguide/java_plugin.html The University of Sydney Page 70 Gradle – Java Plug-in (Project Build) – Java plug-in will add a few tasks – Run gradle tasks to list the tasks of a project – Gradle will compile and create a JAR file containing main classes and resources – run gradle build Example of output of gradle build The University of Sydney Page 71 Gradle – Java Plug-in (Project Build) – clean – Deletes the build directory, removing all built files – assemble – Compiles and jars your code, but does not run the unit tests – Other plugins add more artefacts to this task; – check – Compiles and tests your code – Other plugins add more checks to this task; e.g., checkstyle plugin to run checkstyle against your source code The University of Sydney Page 72 Gradle Java Plug-in – Dependencies https://docs.gradle.org/current/userguide/java_plugin.html – Reference external JAR files that the project is dependent on: – JAR files in a repository (artefacts/dependencies needed for a project) – Different repositories types are supported in Gradle (see Gradle Repository Types) – Example (using Central Maven Repository) production classes have a compile-time dependency on commons collections Test classes have a compile-time dependency on junit https://docs.gradle.org/current/userguide/java_plugin.html https://docs.gradle.org/current/userguide/repository_types.html The University of Sydney Page 73 Gradle – Java Plug-in (Project Customization) https://docs.gradle.org/current/userguide/java_plugin.html – The Java plug-in adds many properties with default values to a project – Customize default values to suit project needs – Use Gradle properties to list properties of a project https://docs.gradle.org/current/userguide/java_plugin.html The University of Sydney Page 74 Gradle – Java Plug-in (Publish JAR file) https://docs.gradle.org/current/userguide/java_plugin.html – Artefacts such as JAR files can be published to repositories – To publish a JAR file – gradle uploadArchives Publish a JAR file to a local repository https://docs.gradle.org/current/userguide/java_plugin.html The University of Sydney Page 75 Gradle – Complete Build file for Java https://docs.gradle.org/current/userguide/java_plugin.html Eclipse plug-in to create the Eclipse-specific descriptor files, like .project https://docs.gradle.org/current/userguide/java_plugin.html The University of Sydney Page 79 References – Ian Sommerville 2016. Software Engineering: Global Edition (3rd edition). Pearson, Englad – Wikipedia, Apache Ant, https://en.wikipedia.org/wiki/Apache_Ant – Wikipedia, Apache Maven, https://en.wikipedia.org/wiki/Apache_Maven – Apache Maven, https://maven.apache.org/ – Gradle documentation, https://docs.gradle.org/current/userguide/ – Tim Berglund and Matthew McCullough. 2011. Building and Testing with Gradle (1st ed.). O'Reilly Media, Inc. https://en.wikipedia.org/wiki/Apache_Ant https://en.wikipedia.org/wiki/Apache_Maven https://maven.apache.org/ https://docs.gradle.org/current/userguide/java_plugin.html