Compiler Theory
Assignment 4, January 31, 2019
The LEX Lexical Analyzer Generator
There is a famous software program called LEX that generates a lexical analyzer
program in C using regular expressions provided by the user. The generated C
file is called lex.yy.c by default. It can run by itself, or be incorporated
into another program.
Reading: The textbook, section 3.7.
1. Install Flex: Copy the folder \\hopper\acad\math\Compiler\Flex to your
computer, in a folder, say z:\Flex. Note: There are various forms of the
Flex documentation in the doc subfolder. It’s pretty short; I suggest you
read through section 0.7 or so.
2. Launch Visual Studio, New Project, Win32 Console Application, empty project.
Name: flex-assn
Location: C:\compiler (or whereever)
Application settings: Empty project.
3. In Solution Explorer, right-click “Source Files”, Add, New, Code.
Name: flex-demo.lex
4. Copy the LEX code at the end of this document into the file.
5. Visual Studio has no clue what to do with a *.lex file, so we have to tell it.
In Solution Explorer, right-click on flex-demo.lex, Properties.
a. In the General panel, click on the line “Excluded from Build” and use the
pull-down arrow at the right of the line to select “No”.
b. Click on the “Item Type” line, and select “Custom Build Tool”.
c. Click “Apply” at the bottom right.
d. In the left panel, click on “Custom Build Tool”.
e. In the “Command Line” box on the right, put the path to the flex program
and the name of your flex input file, something like:
z:\Flex\flex flex-demo.lex
f. In the “Outputs” box, put “lex.yy.c” (without the quotes).
g. Set “Link Objects” to “No”.
h. OK to close the Properties window.
6. In the “Build” menu, click “Compile”. The output window should produce
any error messages from flex. Fix errors, until you get “Build succeeded”.
7. In Solution Explorer, right-click on “Source Files”, Add, Existing Item,
and select lex.yy.c.
8. The analyzer program needs to link with the library flexlib.lib, which is
part of the flex distribution, so we have to tell Visual Studio to do that.
In Solution Explorer, right-click on the project name, Properties.
In the left panel, expand “Linker” and select “Input”. Highlight the
“Additional Dependencies” line, and use the pull-down arrow at the
right to select “
“z:\Flex\flexlib.lib”, using your flex directory instead of “z:\Flex”.
Ok.
9. Still in the Properties/Linker/Input window, we have to fix up one Visual
Studio quirk. The standard C library is called libc.lib, but Visual
Studio 15 decided to change its name. So in the “Ignore Specific Default
Libraries” put “libc.lib”.
10. Do Build/Build Solution to compile the lexical analyzer.
OVER
11. Now run the lexical analyzer, Debug/Start Without Debugging. It will
launch a blank console window. Type in some input, like
123 + 234 * 23 – adfas / begin 123.456
And CTRL-Z ENTER to exit.
Note: You could avoid all that Visual Studio stuff by using a terminal window.
Just three commands:
z:\Flex\flex flex-demo.lex
cl lex.yy.c z:\Flex\flexlib.lib /link /NODEFAULTLIB:”libc.lib”
lex.yy
Assignment:
1. Add these tokens: = == < <= > >= ( ) ! !=
2. Add scientfic notation numbers to the flex analyzer, such as
0.2234e4 122.323E-13
To submit, copy your project folder to hopper and email me.
FLEX source code:
%{
/* flex-demo.lex Sample LEX code */
/* This section is copied to the top of the output C file. */
/* Note: the rest of the line after %{ is NOT copied. */
#include
#include
#include
%}
/* Some handy abbreviations */
DIGIT [0-9]
ID [a-zA-Z_][a-zA-Z0-9_]*
%%
/* Rules section. Patterns and actions. Patterns NOT indented!! */
/* But note this comment is indented, so it is not mistaken for a pattern. */
/* No spaces in the patterns, except when used as an input character. */
{DIGIT}+ { printf( “An integer: %s (%d)\n”, yytext, atoi( yytext ) ); }
{DIGIT}+”.”{DIGIT}* { printf( “A float: %s (%g)\n”, yytext, atof( yytext ) ); }
if|then|begin|end|procedure|function { printf( “A keyword: %s\n”, yytext ); }
{ID} { printf( “An identifier: %s\n”, yytext ); }
“+”|”-“|”*”|”/” { printf( “An operator: %s\n”, yytext ); }
[ \t\n]+ { /* no action; just eat up whitespace */ }
. { /* dot matches any non-newline character */
printf( “Unrecognized character: %s\n”, yytext );
}
%%
// This section copied to the end of the generated C file.
main( )
{ yyin = stdin; // Tell analyzer to use standard input
yylex(); // start the analyzer
}