Week 4: Compiling your models – code generation
Dr Steffen Zschaler
Session goal
Copyright By PowCoder代写 加微信 powcoder
Today, we build a simple compiler for our DSML
– Learn about template-based code generation
– Build a compiler for our Turtles language to generate executable Java code
Style of learning
– Active work: I will introduce a topic, you will try it out on your machines
– Pair work: Two students to share a computer during the active-work sessions
– Prepared work: building on the work you have done in preparation for the session
07/02/2020 (c) King’s College London 2
Making models execute
The elements of language
Defines meaning of
– Defines what a model means
Abstract syntax
– Language concepts – Class, Attribute, Cell, CellState – Links between these concepts
– Class has Attributes
– Cell has CellStates
– Transitions have source and target CellStates
Represents to user
– How computer interacts with models
07/02/2020 (c) King’s College London
Indirectly defines meaning of
– Graphical, textual, tabular, …
– How humans interact with models
Concrete syntax Concrete syntax
– Graphical, textual, tabular, …
– How humans interact with models
The elements of language
Defines meaning of
Abstract syntax
– Language concepts – Class, Attribute, Cell, CellState – Links between these concepts
– Class has Attributes
– Cell has CellStates
– Transitions have source and target CellStates
Represents to user
– How computer interacts with models
– Defines what a model means
07/02/2020
(c) King’s College London
Indirectly defines meaning of
– Graphical, textual, tabular, …
– How humans interact with models
Concrete syntax Concrete syntax
– Graphical, textual, tabular, …
– How humans interact with models
Executing models
To run a model, we need to say what it means – this is called providing the semantics
– Two main practical ways:
1. Operational semantics: write an interpreter for your language
– Could be a simple Java program interpreting a model
– Declarative approaches exist, we will see some later in the term 2. Translational semantics: write a compiler for your language
– Could translate directly to machine code like a compiler for a programming language – For a DSML that’s often overkill: better to generate code in an existing programming
– This is called code generation
07/02/2020 (c) King’s College London 5
Code Generation
Modelling Language
instance of
07/02/2020 (c) King’s College London 6
Code Generation
Modelling Language
instance of
07/02/2020 (c) King’s College London 6
Code Generation
Modelling Language
instance of
instance of
Text (aka program)
07/02/2020 (c) King’s College London
Programming Language
Code Generation
Modelling Language
instance of
instance of
instance of
Text (aka program)
07/02/2020 (c) King’s College London
Generation Template
Programming Language
Code Generation
Modelling Language
instance of
instance of
“Model to text transformation”
instance of
Text (aka program)
07/02/2020 (c) King’s College London
Generation Template
Programming Language
Model-to-Text Transformation Engines
Quite a few exist
Java Emitter Templates Xtend Template Expressions MOF2Script
Epsilon Generation Language …
All based on the same concept: Templates
17/03/2016 7
Model-to-Text Transformation Engines
Quite a few exist
Java Emitter Templates
MOF2Script
Epsilon Generation Language …
All based on the same concept: Templates
Xtend Template Expressions
17/03/2016 7
What do templates look like?
Code generation is specified using template expressions
– Mix of target text and scripting code to evaluate model contents
– Target text is directly output
– Scripting code is evaluated and replaced by string representation of result
– Scripting code enclosed in «guillemets»
– Typically don’t occur in target language code
– UTF characters – make sure all files are encoded in UTF-8 by Eclipse
– Scripting language: Xtend with some additional control support Templates are then evaluated for specific models or model elements
07/02/2020 (c) King’s College London 8
A template example
07/02/2020 (c) King’s College London 9
A template example
Templates can be defined like any other Xtend method. Alternatively, you can use a template wherever a String expression is expected.
07/02/2020 (c) King’s College London 9
A template example
Three single quotes start and end a template expression.
07/02/2020 (c) King’s College London 9
A template example
Anything not in guill- emets is target text.
07/02/2020 (c) King’s College London 9
A template example
Guillemets enclose expressions that will be replaced by their results.
07/02/2020 (c) King’s College London 9
A template example
Special FOR loops and IF statements allow statements to be broken across multiple scripting blocks.
07/02/2020 (c) King’s College London 9
Guillemets??
All template languages need a way of separating target code and scripting code
– Sometimes called a boundary character
– Should not be something typically used in the target language
– Otherwise we always have to differentiate between using it for opening scripting code and actual target-language use
– Called “escaping”
– Other templating languages use things like [% (EGL) or <% (JET, JSP, ASP) - Xtend uses single-character guillemets
- Pro: Hardly ever used in any programming language - Con: Can be difficult to enter
- Use Ctrl+< in Eclipse
- Con: Can get confusing if files are written and read assuming different encodings
- Always use UTF-8 encoding
07/02/2020 (c) King's College London 10
Control structures in template expressions
Within guillemets, you can put any Xtend expression
- Direct model navigation
- Including using Xtend’s compact ways of navigating through collections using map, reduce, fold
But can also use special statements that can span multiple guillemet blocks
- Keywords are always in all capital letters - Statements supported are
-FOR/ENDFOR – evaluate the template code contained for all elements of a collection, can use BEFORE, AFTER, and SEPARATOR to specify helper strings
-IF/ELSEIF/ELSE/ENDIF – conditionally evaluate template code Alternatively can always replace with nested template expressions
07/02/2020 (c) King's College London 11
Integrating code generation into an Xtext language
Xtext integrates smoothly into Eclipse’s build system
- Code generators can be directly invoked as part of automated, incremental build
- Every time a document in our DSML is saved, code generation for this file and any dependencies
can be triggered
07/02/2020 (c) King's College London 12
Integrating code generation into an Xtext language
Xtext integrates smoothly into Eclipse’s build system
- Code generators can be directly invoked as part of automated, incremental build
- Every time a document in our DSML is saved, code generation for this file and any dependencies
can be triggered
To hook into this
- Edit the generated XYZGenerator stub
07/02/2020 (c) King's College London 12
Integrating code generation into an Xtext language
Xtext integrates smoothly into Eclipse’s build system
- Code generators can be directly invoked as part of automated, incremental build
- Every time a document in our DSML is saved, code generation for this file and any dependencies
can be triggered
To hook into this
- Edit the generated XYZGenerator stub - Implement its doGenerate method
07/02/2020 (c) King's College London 12
Integrating code generation into an Xtext language
Xtext integrates smoothly into Eclipse’s build system
- Code generators can be directly invoked as part of automated, incremental build
- Every time a document in our DSML is saved, code generation for this file and any dependencies
can be triggered
To hook into this
- Edit the generated XYZGenerator stub - Implement its doGenerate method
- Resource is EMF’s way of referring to a file that contains a parsed model
- contents is a list of EObjects; roots of the model(s) contained
- This resource is the one for which we’re asked to generate code.
07/02/2020 (c) King's College London 12
Integrating code generation into an Xtext language
Xtext integrates smoothly into Eclipse’s build system
- Code generators can be directly invoked as part of automated, incremental build
- Every time a document in our DSML is saved, code generation for this file and any dependencies
can be triggered
To hook into this
- Edit the generated XYZGenerator stub - Implement its doGenerate method
- IFileSystemAccess2 is Xtext’s way of giving access to Eclipse’s file system
- Allows to generate file names and resources for generation targets
- These will normally live in a src-gen/ folder created in 0t7h/0e2/2s0o20urc(ec) rKeinsg'soCuolrlecgeL’osndponroject
Integrating code generation into an Xtext language
Xtext integrates smoothly into Eclipse’s build system
- Code generators can be directly invoked as part of automated, incremental build
- Every time a document in our DSML is saved, code generation for this file and any dependencies
can be triggered
To hook into this
- Edit the generated XYZGenerator stub - Implement its doGenerate method
- IGeneratorContext gives access to Eclipse’s job framework
- Primarily provides a CancelIndicator to allow long- running generator jobs to be aborted by the user
07/02/2020 (c) King's College London 12
Standard structure of an Xtext generator
07/02/2020 (c) King's College London 13
Standard structure of an Xtext generator
Extract the actual model from the resource
- Usually, only one object, this will be the first in the list – the list head
- Cast to the correct type depending on your language’s metamodel
07/02/2020 (c) King's College London 13
Standard structure of an Xtext generator
Use filesystem access to generate target file.
May have multiple such calls, one for each file to be generated.
07/02/2020 (c) King's College London 13
Standard structure of an Xtext generator
Generate a name for the target file
- Can be a relative path
- Name should depend on name of original file (hence
providing the resource) or some model contents
07/02/2020 (c) King's College London 13
Standard structure of an Xtext generator
07/02/2020 (c) King's College London
By default, we’re using the file name of the original file
Standard structure of an Xtext generator
Call the actual generation template
- Here we are using Xtend’s extension-method mechanism
07/02/2020 (c) King's College London 13
Standard structure of an Xtext generator
Template goes here
Remember, this is just a method, so can call other methods
→use to modularise your templates
07/02/2020 (c) King's College London 13
In your pairs
- Implement a simple code generator for your turtles language - Integrate it into the Eclipse auto-build system
- For each turtles program
- Generate a .txt file of the same name as the original file
- Include text showing the number of commands of different types
You have 15 minutes
Download last step as tag DSML_v7_scopes
from the Turtle DSML repository linked from KEATS
31/01/2020 (c) King's College London 14
Testing code generation
Xpect supports testing code generators
07/02/2020 (c) King's College London 17
Testing code generation
Xpect supports testing code generators
- Use XPECT generated command at beginning of xt file (but after XPECT_SETUP)
07/02/2020 (c) King's College London 17
Testing code generation
Xpect supports testing code generators
- Use XPECT generated command at beginning of xt file (but after XPECT_SETUP) - Need to make some adjustments to XPECT test project:
- Add org.eclipse.xtext.xbase.lib to the plugin dependencies
07/02/2020 (c) King's College London 17
Testing code generation
Xpect supports testing code generators
- Use XPECT generated command at beginning of xt file (but after XPECT_SETUP) - Need to make some adjustments to XPECT test project:
- Add org.eclipse.xtext.xbase.lib to the plugin dependencies
- Change the test class to extend GeneratorTest (and include other types of tests via
@XpectSuiteClasses)
07/02/2020 (c) King's College London 17
Testing code generation
Xpect supports testing code generators
- Use XPECT generated command at beginning of xt file (but after XPECT_SETUP) - Need to make some adjustments to XPECT test project:
- Add org.eclipse.xtext.xbase.lib to the plugin dependencies
- Change the test class to extend GeneratorTest (and include other types of tests via
@XpectSuiteClasses)
Now we can develop code generators in a test-driven way
07/02/2020 (c) King's College London 17
In your pairs
- Write a simple test for your simple turtles code generator
- Run the test. It should go through.
- Extend the test to also expect the count of var statements to be generated. - Run the extended test. It should break.
- Fix the code generator to make the test work again.
You have 10 minutes
Download last step as tag DSML_v8_stats_generator from the Turtle DSML repository linked from KEATS
31/01/2020 (c) King's College London 18
Real code generation – how to build it?
Code generation is all about understanding variability
How do solution implementations in your domain
- Share the same code→put this code into a library or framework - Differ→generate this code against the library / framework
07/02/2020 (c) King's College London 20
Real code generation – how to build it?
Code generation is all about understanding variability
How do solution implementations in your domain
- Share the same code→put this code into a library or framework - Differ→generate this code against the library / framework
Modelling Language
instance of
instance of
instance of
Text (aka program)
07/02/2020 (c) King's College London
Generation Template
Programming Language
Real code generation – how to build it?
Code generation is all about understanding variability
How do solution implementations in your domain
- Share the same code→put this code into a library or framework - Differ→generate this code against the library / framework
Modelling Language
Generation Template
Programming Language
instance of
instance of
instance of
instance of
Text (aka program)
Library/Framework
07/02/2020 (c) King's College London
Real code generation – how to build it?
Code generation is all about understanding variability
How do solution implementations in your domain
- Share the same code→put this code into a library or framework - Differ→generate this code against the library / framework
instance of instance of
instance of
instance of
Modelling Language
Model Text (aka program)
Domain variability
07/02/2020 (c) King's College London
Generation Template
Programming Language
Library/Framework
An easy recipe for building a code generator
Building code generators is easiest if you have examples
- Each a combination of
- A concrete model in the DSML
- Code you would expect to be generated for this specific model
07/02/2020 (c) King's College London 21
An easy recipe for building a code generator
Building code generators is easiest if you have examples
- Each a combination of
- A concrete model in the DSML
- Code you would expect to be generated for this specific model
Then work backwards:
- Mark all places that need to change depending on model information
07/02/2020 (c) King's College London 21
An easy recipe for building a code generator
Building code generators is easiest if you have examples
- Each a combination of
- A concrete model in the DSML
- Code you would expect to be generated for this specific model
Then work backwards:
- Mark all places that need to change depending on model information - Replace each marked place with appropriate scripting code
07/02/2020 (c) King's College London 21
Tests for code generators that produce multiple files
Code generators can produce multiple files
- Xpect tests can test each file separately
- Can include multiple XPECT generated statements
07/02/2020 (c) King's College London 22
Tests for code generators that produce multiple files
Code generators can produce multiple files
- Xpect tests can test each file separately
- Can include multiple XPECT generated statements - Use the file parameter for differentiation
07/02/2020 (c) King's College London 22
Use file to specify the expected name of the generated file
Xtend technical tangent: dispatch methods and the visitor pattern
Processing models often requires adding new operations to nodes in the abstract syntax
- For example a generate operation producing Java code corresponding to any abstract-syntax concept
- Need same signature but different implementations for different concepts - Very difficult to do in standard OO languages
- Typically addressed using the Visitor pattern [1]
[1] Gamma, Erich; Helm, Richard; Johnson, Ralph; Vlissides, John (1995). Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley. ISBN 978-0-201-63361-0.
07/02/2020 (c) King's College London 23
Xtend technical tangent: dispatch methods and the visitor pattern
Processing models often requires adding new operations to nodes in the abstract syntax
- For example a generate operation producing Java code corresponding to any abstract-syntax concept
- Need same signature but different implementations for different concepts - Very difficult to do in standard OO languages
- Typically addressed using the Visitor pattern [1]
Xtend provides direct support for double-dispatch extension methods:
- Extension method: a method declared in one place that behaves as if it was declared as a method of its first argument
- Double-dispatch method: multiple extension methods with same signature, only differing in type of first argument
- Choice is made at runtime depending on runtime type of argument
- Works exactly like a polymorphic method defined outside of the containing class
[1] Gamma, Erich; Helm, Richard; Johnson, Ralph; Vlissides, John (1995). Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley. ISBN 978-0-201-63361-0.
07/02/2020 (c) King's College London 23
Xtend technical tangent: dispatch methods and the visitor pattern
Processing models often requires adding new operations to nodes in the abstract syntax
- For example a generate operation producing Java code corresponding to any abstract-syntax concept
- Need same signature but different implementations for different concepts
- Very difficult to do in standard OO languages
Use of an extension method: see how and where doGenerate is declared.
- Typically addressed using the Visitor pattern [1]
Notice change in syntax colouring to flag that this is an extension method.
Xtend provides direct support for double-dispatch extension methods:
- Extension method: a method declared in one place that behaves as if it was declared as a method of its first argument
- Double-dispatch method: multiple extension methods with same signature, only differing in type of first argument
- Choice is made at runtime depending on runtime type of argument
- Works exactly like a polymorphic method defined outside of the containing class
[1] Gamma, Erich; Helm, Richard; Johnson, Ralph; Vlissides, John (1995). Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley. ISBN 978-0-201-63361-0.
07/02/2020 (c) King's College London 23
Xtend technical tangent: dispatch methods and the visitor pattern
Processing models often requires adding new operations to nodes in the abstract syntax
- For example a generate operation producing Java code corresponding to any abstract-syntax concept
- Need same signature but different implementations for different concepts - Very difficult to do in standard OO languages
- Typically addressed using the Visitor pattern [1]
Xtend provi
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com