Agile Software
Development Practices SOF2412 / COMP9412 System Build Automation
Dr. Basem Suleiman
School of Information Technologies
The University of Sydney
Page 1
Agenda
– Software Configuration Management – SystemBuilding
– AgileSystemBuild
– Software Build Automation Tools – Ant
– Maven – Gradle
The University of Sydney
Page 2
Software Configuration Management
The University of Sydney Page 3
•
Revisit – Software Development Process Model
Rational Unified Process (RUP) iterative and incremental approach to develop OO software systems
Requirements Design
Implementation & Test & Integration & More Design
Final Integration & System Test
3 weeks (for example)
Requirements Time Design
Implementation & Test & Integration & More Design
Final Integration & System Test
Feedback from iteration N leads to refinement and adaptation of the requirements and design in iteration N+1.
The University of Sydney
Page 4
Iterations are fixed in length, or timeboxed.
The system grows incrementally.
Agile Development – Increments
• Software is incrementally developed
Work for the current iteration
Every 4 weeks, produce a new shippable product
Work for future iterations
Agile Vs Waterfall: Showdown For Software Development Domination
The University of Sydney Page 5
Configuration Management (CM)
– Configuration management (CM) is concerned with the policies, processes and tools for managing changing software systems
– Track of what changes and component versions incorporated into each system version
– Essential for team projects to control changes made by different developers
The University of Sydney Page 6
Configuration Management Activities
The University of Sydney Page 7
Configuration Management Activities
– System building: assembling program components, data and libraries, then compiling these to create an executable system
– Version management: keeping track of the multiple versions of system components and ensuring that changes made to components by different developers do not interfere with each other
– Change management: keeping track of requests for changes to the software from customers and developers, working out the costs and impact of changes, and deciding the changes should be implemented
– Release management: preparing software for external release and keeping track of the system versions that have been released for customers
The University of Sydney Page 8
Multi-version System Development
Ian Sommerville. 2016. Software Engineering (10th ed.)
The University of Sydney Page 9
Version Management (VM)
– Keep track of different versions of software components or configuration items and the systems in which these components are used
– Ensuring changes made by different developers to these versions do not interfere with each other
– The process of managing code-lines and baselines
The University of Sydney
Page 11
Baselines
Ian Sommerville. 2016. Software Engineering (10th ed.)
The University of Sydney Page 12
Version Management – Codelines and Baselines
– Version Management can be thought of as the process of managing codelines and baselines
– Codeline: a sequence of versions of source code with later versions in the sequence derived from earlier versions
– System’scomponentsoftenhavedifferentversions
– Baseline: a definition of a specific system
– Specifies the component versions that are included in the system plus a specification of the libraries used, configuration files, etc.
The University of Sydney
Page 13
Baselines
– Baselines may be specified using a configuration language, which allows you to define what components are included in a version of a particular system.
– Baselines are important because you often have to recreate a specific version of a complete system
– E.g., individual system versions for different customers. If a customer reports bugs in their system one can recreate the version delivered to a specific customer
The University of Sydney Page 14
–
Semantic Versioning (SemVer)
– –
Set of rules and requirements that determine how version numbers should be assigned and incremented for software being developed
– Semantic numbers; numbers with meaning in relation to a certain version
Why SemVer?
– Helps managing versioning numbers in a meaningful and standard way
– Managing dependencies: the bigger your system grows, the more packages/libraries/plugins you integrate into your software
Given a version number MAJOR.MINOR.PATCH, increment the:
1. MAJOR version when you make incompatible API changes,
2. MINOR version when you add functionality in a backwards-compatible manner, 3. PATCH version when you make backwards-compatible bug fixes.
https://semver.org/
The University of Sydney
Page 16
Semantic Versioning – Example
Stage
Code
Rule
Example
New product
1st release
Start with 1.0.0
1.0.0
Patch Release
Bug fixes, other minor changes
Increment the 3rd digit
1.0.1
Minor Release
New features that do not break existing features
Increment the 2nd digit
1.1.0
Major Release
Changes that break backward compatibility
Increment the 1st digit
2.0.0
https://docs.npmjs.com/getting-started/semantic-versioning More details on semantic versioning – https://semver.org/
The University of Sydney Page 17
SemVer Example – Dependencies
– “Firetruck” library requires a semantically versioned package “Ladder”
– Firetruck uses Ladder 3.1.0.
– Assume that Ladder3.1.9 is released, can you safely use it safely to release it in your system?
– Assume that Ladder3.7.7 is released, can you safely use it safely to release it in your system?
– Assume that Ladder4.1.0 is released, can you safely use it safely to release it in your system?
https://semver.org/#semantic-versioning-specification-semver
The University of Sydney Page 18
Agile Development in CM
– Agile development, where components and systems are changed several times per day, is impossible without using CM tools
– The definitive versions of components are held in a shared project repository and developers copy these into their own workspace
– They make changes to the code then use system building tools to create a new system on their own computer for testing. Once they are happy with the changes made, they return the modified components to the project repository.
The University of Sydney Page 19
System Building
The University of Sydney Page 20
System Building
– System building is the process of creating a complete, executable system by compiling and linking the system components, external libraries, configuration files, etc.
– System building tools and version management tools must communicate as the build process involves checking out component versions from the repo managed by the version management system.
– The configuration description used to identify a baseline is also used by the system building tool
The University of Sydney Page 21
Software Build Automation Tools
Ant, Maven, Gradle
The University of Sydney Page 27
Tools for Agile Development
The University of Sydney
Page 28
Version Control
Build Automation
Build Tools – Apache Ant
– Java-based software build tool for automating build processes – Requires Java platform and best suited for building Java projects
– Does not impose coding conventions
– Does not impose any heavyweight dependency management framework
– XML to describe the code build process and its dependencies
– Defaultbuild.xml
The University of Sydney
Page 29
Apache ANT – Example
https://en.wikipedia.org/wiki/Apache_Ant
The University of Sydney Page 30
Apache ANT – Drawbacks
– Too flexible
– Complexity (XML-based build files)
– Need to specify a lot of things to make simple builds – No standard structure/layout
– Developers can create their own structure/layout of the project
The University of Sydney
Page 31
Apache Maven
– A build automation tool primarily for java projects
– XML-based description of the software being built
– Dependencies on other external modules and components, the build order, directories, and required plug-ins
– Central repository (e.g., Maven 2)
– Coding by convention: it uses conventions over configuration for the build
procedure
– Supported by Eclipse, IntelliJ, JBuilder, NetBeans
– Plugin-based architecture
– Plugin for the .NET framework and native plugins for C/C++ are maintained
The University of Sydney
Page 32
Apache Maven – Minimal Example
– Maven projects are configured using Project Object Model (POM) stored in a pom.xml file
The University of Sydney
Page 33
Apache Maven – Project Structure
– The command mvn package will
– compile all the Java files
– run any tests
– package the deliverable code and resources into target
(e.g., target/my-app-1.0.jar)
The Maven software tool auto-generated this directory structure for a Java project
https://commons.wikimedia.org/wiki/File:Maven_CoC.svg#/media/File:Maven_CoC.svg
The University of Sydney Page 34
Apache Maven – Central Repository
– Maven uses default Central Repository that maintains required software artefacts (libraries, plug-ins) to manage dependencies
– E.g., project that is dependent on the Hibernate library needs to specify that in the pom.xml project file
– Maven checks if the referenced dependency is already in the user’s local repository
– It references the dependency from the local repository or
– Dynamically download the dependency and the dependencies that Hibernate itself
needs (transitive dependency) and store them in the user’s local repository
– You can configure repositories other than the default (e.g., company- private repository)
The University of Sydney Page 35
Apache Maven – Drawbacks
– Again, XML-based files increase complexity (verbose)
– Rigid; developers are required to understand follow the conventions
The University of Sydney Page 36
Gradle
– Build automation tool that builds upon the concepts of Ant and Maven
– build conventions, and redefine conventions
– Project described using Groovy-based Domain Specific Language (DSL)
– Tasks orders determined using a directed acyclic graph (DAC)
– Multi-project builds
– Incremental builds; up to date parts of the build tree do not need to be re- executed
– Dependency handling (transitive dependency management)
https://en.wikipedia.org/wiki/Gradle
The University of Sydney
Page 37
Gradle – Groovy
– Gradle build files are Groovy scripts
– Groovy is a dynamic language of the JVM
– Canbeaddedasaplug-in
– Allowsdeveloperstowritegeneralprogrammingtasksinthebuildfiles
– ReliefdevelopersfromthelackingcontrolflowinAntorbeingforcedinto plug-in development in Maven to declare nonstandard tasks
https://en.wikipedia.org/wiki/Gradle
The University of Sydney Page 38
Gradle – DSL
– Gradle also presents a DSL tailored to the task of building code – Notgeneral-purposeorprogramminglanguage
– GradleDSLcontainsthelanguageneededtodescribehowtobuildJava code and create a WAR file from the output
– Gradle DSL is extensible through plug-ins
https://en.wikipedia.org/wiki/Gradle
The University of Sydney Page 39
Gradle – Extensible DSL
– Extensibility using plug-ins (if it doesn’t have the language to describe what you want your build to do)
– E.g., describe how to run database migration scripts or deploy code to a set of cloud-based QA servers
– Gradleplug-insallow:
– Addingnewtaskdefinitions
– Changethebehaviorofexistingtasks
– Addnewobjects
– CreatekeywordstodescribetasksthatdepartfromthestandardGradle categories
The University of Sydney Page 40
Gradle Basics
The University of Sydney Page 41
Gradle – Tasks
– Task: a single atomic piece of work for a build – e.g., compiling classes
– Project: a composition of several tasks – e.g., Creation of a jar file
– Each task has a name, which can be used to refer to the task within its owning project, and a fully qualified path, which is unique across all tasks in all projects
https://en.wikipedia.org/wiki/Grale
The University of Sydney Page 42
Gradle – Task Actions
– A task is made up of sequence of Action objects – Action.execute(T) to execute a task
– Add actions to a task
– Task.doFirst() or Task.doLast()
– Task action exceptions
– StopActionExceptiontoabortexecutionoftheaction
– StopExecutionException to abort execution of the task and continue to the next task
https://en.wikipedia.org/wiki/Grale
The University of Sydney Page 43
Gradle – Simplest Build File Example
Build.gradle
Build.gradle
task hello << {
print 'hello, '
}
task world(dependsOn: hello) << { println 'world'
}
task helloWorld << { println 'hello, world'
}
Results of Hello World build file
$ gradle -q helloWorld hello, world
execute the second task, world
$ gradle -q world hello, world
The University of Sydney
Page 44
Gradle – Simplest Build File for Java Example (1)
Simplest possible Gradle file for java The University of Sydney
Page 45
Project layout of HelloWorld.java
Gradle – Simplest Build File for Java Example (2)
gradle build
Gradle automatically introduces number of tasks for us to run
Note:
• Class files generated and place in a directory
• Test report files (for unit test results)
• JAR built using the project directory
Run HelloWorld Java
$ java -cp build/classes/main/ org.gradle.example.simple.HelloWorld hello, world
Project Layout of Hello World Java After Build
The University of Sydney
Page 46
Gradle – Build Lifecycle
– Phases of executing a build file in gradle:
– Initialization: projects are to participate in the build
– Configuration:taskobjectsareassembledintoaninternalobjectmodel called Directed Acyclic Graph (DAG)
– Execution: build tasks are executed in the order required by their dependency relationship
The University of Sydney
Page 47
Gradle – Task Configuration
– Configuration block: to setup variables and data structures needed by the task action when it runs in the build
– Make tasks rich object model populated with information about the build
– Runs during gradle’s configuration lifecycle when task actions executed
– Closure: a block of code specified by curly braces
– Holdingblocksofconfigurationandbuildactions
The University of Sydney
Page 48
Gradle – Task Configuration Example
Task
Task actions Configuration block
The University of Sydney
Page 49
Gradle – Tasks are Objects
– Every task is represented internally as an object
– Task’smethodsandproperties
– Gradlecreatesaninternalobjectmodelofthebuildbeforeexecutingit
– Each new task is of DefaultTask type - task type can be changed
– DefaultTaskcontainsfunctionalityrequiredforthemtointerfacewith Gradle project model
The University of Sydney Page 50
Gradle – Methods of Default Task
Method
Description
dependsOn(task)
Adds a task as a dependency of the calling task. A depended-on task will always run before the task that depends on it
doFirst(closure)
Adds a block of executable code to the beginning of a task’s action. During the execution phase, the action block of every relevant task is executed.
doLast(closure)
Appends behavior to the end of an action
onlyIf(closure)
Expresses a predicate which determines whether a task should be executed. The value of the predicate is the value returned by the closure.
The University of Sydney Page 51
Gradle – dependsOn() Example
Different ways to call the dependsOn method
The University of Sydney Page 52
Gradle – doFirst() Example
OR
Call the doFirst on the task object (top) and inside task’s configuration block (bottom)
The University of Sydney
Page 53
Repeated calls to the doFirst method are additive.
Gradle – onlyIf() Example
Using onlyIf method to do simple system property tests
The University of Sydney
Page 54
– –
onlyIf method can be used to switch individual tasks on and off using any logic you can express in Groovy code
E.g., read files, call web services, check security credentials
Gradle – Default Task’s Properties
Method
Description
didWork
A Boolean property indicating whether the task completed successfully
enabled
A Boolean property indicating whether the task will execute.
path
A string property containing the fully qualified path of a task (levels; DEBUG, INFO, LIFECYCLE, WARN, QUIET, ERROR)
logger
A reference to the internal Gradle logger object
logging
The logging property gives us access to the log level
temporaryDir
Returns a File object pointing to a temporary directory belonging to this build file. It is generally available to a task needing a temporary place in to store intermediate results of any work, or to stage files for processing inside the task
description
a small piece of human-readable metadata to document the purpose of a task
The University of Sydney Page 55
Gradle – Dynamic Properties
– Properties (other than built-in ones) can be assigned to a task
– A task object functions can contain other arbitrary property names and values
we want to assign to it (do not use built-in property names)
The University of Sydney Page 56
Gradle Task Types – Copy
– A copy task copies files from one place into another
task copyFiles(type: Copy) { from 'resources'
into 'target'
include '**/*.xml', '**/*.txt', '**/*.properties'
}
The University of Sydney
Page 58
Note: the from, into, and include methods are inherited from the Copy
Gradle Task Types – Jar
– A Jar task creates a jar file from source files
– The Java plug-in’s jar creates a task of this type
– It packages the main source set and resources together with a trivial manifest into a jar bearing the project’s name in the build/libs directory
– highly customizable
The University of Sydney Page 60
Gradle Task Types – Jar Example
apply plugin: 'java'
task customJar(type: Jar) {
manifest {
attributes firstKey: 'firstValue', secondKey: 'secondValue' }
archiveName = 'hello.jar'
destinationDir = file("${buildDir}/jars")
from sourceSets.main.classes
}
The University of Sydney Page 61
Gradle Task Types – JavaExec
– A JavaExec task runs a Java class with a main() method
– Tries to take the hassle away and integrate command-line Java
invocations into your build
The University of Sydney Page 62
Gradle Task Types – JavaExec Example
apply plugin: 'java' repositories { mavenCentral()
}
dependencies {
runtime 'commons-codec:commons-codec:1.5'
}
task encode(type: JavaExec, dependsOn: classes) {
main = 'org.gradle.example.commandline.MetaphoneEncoder' args = "The rain falls mainly in the plain".split().toList() classpath sourceSets.main.classesDir
classpath configurations.runtime
}
The University of Sydney Page 63
Gradle Custom Task Types
– Gradle’s built-in tasks might not be sufficient for all scenarios
– Gradle allows defining custom task types in:
– The Build File and must extend the DefaultTask class or one of its descendants
– The Source Tree: when the task’s logic is significant has its own class hierarchy and might rely on external interface
The University of Sydney
Page 64
• •
When the custom task logic outgrows the build file, it can be migrated to the buildSrc directory at the project root
This directory is automatically compiled and added to the build classpath
Gradle Custom Task Types – Build File
– Suppose your build file needs to issue arbitrary queries against a MySQL
database
* *
*
*
Task’s properties (Groovy idiom)
Task method will run when the task runs
* Actual build tasks inherits MySQLType’s properties and actions
The University of Sydney
Page 65
Gradle Custom Task Types – Source Tree
–.
Similar to the code in the build script (previous example. However, source tree provides a robust way for elaborating on that simple task behavior, growing an object model, writing tests, and doing everything else we normally do when developing software
The University of Sydney
Page 67
Gradle – Plug-ins
Java Plug-in
The University of Sydney Page 69
Gradle – Java Plug-in
– A plug-in is an extension to Gradle which configures projects
– Java plug-in adds some tasks to your project which will compile and unit
test your Java source code, and bundle into a JAR
– Convention-based; default values for many aspects of the project are pre- defined
• In your build.gradle: apply plugin : ‘java’
– Can customize projects if you do not follow the convention
The University of Sydney
Page 70
Gradle – Java Plug-in (Project Structure)
– Gradle expects to find production source code under src/main/java and test source code under src/test/java
– Files under src/main/resources will be included in the JAR as resources
– Files under src/test/resources will be included in the classpath used to run
tests
– All output files are created under the build directory, with the JAR file will
end up in the build/libs directory
https://docs.gradle.org/current/userguide/java_plugin.html
The University of Sydney
Page 71
Gradle – Java Plug-in (Project Build)
– Java plug-in will add a few tasks
– Run gradle tasks to list the tasks of a project
– Gradle will compile and create a JAR file containing main classes and resources – run gradle build
The University of Sydney
Page 72
Example of output of gradle build
Gradle – Java Plug-in (Project Build)
– clean
– Deletes the build directory, removing all built files
– assemble
– Compiles and jars your code, but does not run the unit tests – Other plugins add more artefacts to this task;
– check
– Compiles and tests your code
– Other plugins add more checks to this task; e.g., checkstyle plugin to run checkstyle against your source code
The University of Sydney
Page 73
Gradle Java Plug-in – Dependencies
– Reference external JAR files that the project is dependent on:
– JAR files in a repository (artefacts/dependencies needed for a project)
– Different repositories types are supported in Gradle (see Gradle Repository Types)
– Example (using Central Maven Repository)
production classes have a compile-time dependency on commons collections
Test classes have a compile-time dependency on junit
https://docs.gradle.org/current/userguide/java_plugin.html
The University of Sydney Page 74
Gradle – Java Plug-in (Project Customization)
– The Java plug-in adds many properties with default values to a project
– Customize default values to suit project needs
– Use Gradle properties to list properties of a project
https://docs.gradle.org/current/userguide/java_plugin.html
The University of Sydney
Page 75
Gradle – Java Plug-in (Publish JAR file)
– Artefacts such as JAR files can be published to repositories
– To publish a JAR file
– gradle uploadArchives
https://docs.gradle.org/current/userguide/java_plugin.html
Publish a JAR file to a local repository
The University of Sydney
Page 76
Gradle – Complete Build file for Java
Eclipse plug-in to create the Eclipse-specific descriptor files, like .project
https://docs.gradle.org/current/userguide/java_plugin.html
The University of Sydney Page 77
References
– Ian Sommerville 2016. Software Engineering: Global Edition (3rd edition). Pearson, Englad
– Wikipedia, Apache Ant, https://en.wikipedia.org/wiki/Apache_Ant
– Wikipedia, Apache Maven, https://en.wikipedia.org/wiki/Apache_Maven
– Apache Maven, https://maven.apache.org/
– Gradle documentation, https://docs.gradle.org/current/userguide/
– Tim Berglund and Matthew McCullough. 2011. Building and Testing with Gradle (1st ed.). O'Reilly Media, Inc.
The University of Sydney
Page 81