Stream Editor
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-1
Topic 6
SED – STREAM EDITOR
What is sed?
A non-interactive stream editor
Interprets sed instructions and performs actions
Use sed to:
◦ Automatically perform edits on file(s)
◦ Simplify doing the same edits on multiple files
◦ Write conversion programs
2
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-2
The sed command
3
TOPIC 6 – SED – STREAM EDITOR
sed command syntax
4
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-3
sed Operation
5
T
O
P
IC
6
–
S
E
D
–
S
T
R
E
A
M
E
D
IT
O
R
How Does sed Work?
sed reads line of input
◦ line of input is copied into a temporary buffer called pattern space
◦ editing commands are applied
◦ subsequent commands are applied to line in the pattern space, not the
original input line
◦ once finished, line is sent to output
(unless –n option was used)
◦ line is removed from pattern space
sed reads next line of input, until end of file
Note: input file is unchanged
6
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-4
sed instruction format
address determines which lines in the input file are to be processed by
the command(s)
◦ if no address is specified, then the command is applied to each input line
address types:
◦ Single-Line address
◦ Set-of-Lines address
◦ Range address
◦ Nested address 7
TOPIC 6 – SED – STREAM EDITOR
Single-Line Address
Specifies only one line in the input file
◦ special: dollar sign ($) denotes last line of input file
Examples:
◦ show only line 3
sed -n -e ‘3 p’ input-file
◦ show only last line
sed -n -e ‘$ p’ input-file
◦ substitute “endif” with “fi” on line 10
sed -e ’10 s/endif/fi/’ input-file
8
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-5
Set-of-Lines Address
use regular expression to match lines
◦ written between two slashes
◦ process only lines that match
◦ may match several lines
◦ lines may or may not be consecutives
Examples:
sed -e ‘/key/ s/more/other/’ input-file
sed -n -e ‘/r..t/ p’ input-file
9
TOPIC 6 – SED – STREAM EDITOR
Range Address
Defines a set of consecutive lines
Format:
start-addr,end-addr (inclusive)
Examples:
10,50 line-number,line-number
10,/R.E/ line-number,/RegExp/
/R.E./,10 /RegExp/,line-number
/R.E./,/R.E/ /RegExp/,/RegExp/
10
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-6
Example: Range Address
% sed -n -e ‘/^BEGIN$/,/^END$/p’ input-file
Print lines between BEGIN and END, inclusive
BEGIN
Line 1 of input
Line 2 of input
Line3 of input
END
Line 4 of input
Line 5 of input
11
addr1 addr2
These lines are
printed
TOPIC 6 – SED – STREAM EDITOR
Nested Address
Nested address contained within another address
Example:
print blank lines between line 20 and 30
20,30{
/^$/ p
}
12
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-7
Address with !
address with an exclamation point (!):
instruction will be applied to all lines that do not match the address
Example:
print lines that do not contain “obsolete”
sed -e ‘/obsolete/!p’ input-file
13
TOPIC 6 – SED – STREAM EDITOR
sed commands
14
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-8
Line Number
line number command (=) writes the current line number before each
matched/output line
Examples:
sed -e ‘/Two-thirds-time/=’ tuition.data
sed -e ‘/^[0-9][0-9]/=’ inventory
15
TOPIC 6 – SED – STREAM EDITOR
modify commands
16
Substitute
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-9
Insert Command: i
adds one or more lines directly to the output before the address:
◦ inserted “text” never appears in sed’s pattern space
◦ cannot be used with a range address; can only be used with the single-line
and set-of-lines address types
Syntax:
[address] i\
text
17
TOPIC 6 – SED – STREAM EDITOR
Example: Insert Command (i)
% cat tuition.insert.sed
1 i\
Tuition List\
% cat tuition.data
Part-time 1003.99
Two-thirds-time 1506.49
Full-time 2012.29
% sed -f tuition.insert.sed tuition.data
Tuition List
Part-time 1003.99
Two-thirds-time 1506.49
Full-time 2012.29
18
Input data
Sed script to insert “Tuition List”
as report title before line 1
Output after applying
the insert command
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-10
Append Command: a
adds one or more lines directly to the output after the address:
◦ Similar to the insert command (i), append cannot be used with a range
address.
◦ Appended “text” does not appear in sed’s pattern space.
Syntax:
[address] a\
text
19
TOPIC 6 – SED – STREAM EDITOR
Example: Append Command
(a)
% cat tuition.append.sed
a \
————————–
% cat tuition.data
Part-time 1003.99
Two-thirds-time 1506.49
Full-time 2012.29
% sed -f tuition.append.sed tuition.data
Part-time 1003.99
————————–
Two-thirds-time 1506.49
————————–
Full-time 2012.29
————————–
20
Input data
Sed script to append
dashed line after
each input line
Output after applying
the append command
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-11
Change Command: c
replaces an entire matched line with new text
accepts four address types:
◦ single-line, set-of-line, range, and nested addresses.
Syntax:
[address1[,address2]] c\
text
21
TOPIC 6 – SED – STREAM EDITOR
Example: Change Command
(c)
% cat tuition.change.sed
1 c\
Part-time 1100.00
% cat tuition.data
Part-time 1003.99
Two-thirds-time 1506.49
Full-time 2012.29
% sed -f tuition.change.sed tuition.data
Part-time 1100.00
Two-thirds-time 1506.49
Full-time 2012.29
22
Input data
Sed script to change
tuition cost from
1003.99 to 1100.00
Output after applying
the change command
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-12
Delete Command: d
deletes the entire pattern space
◦ commands following the delete command are ignored since the deleted text
is no longer in the pattern space
Syntax:
[address1[,address2]] d
23
TOPIC 6 – SED – STREAM EDITOR
Example: Delete Command (d)
Remove part-time data from “tuition.data” file
% cat tuition.data
Part-time 1003.99
Two-thirds-time 1506.49
Full-time 2012.29
% sed –e ‘/^Part-time/d’ tuition.data
Two-thirds-time 1506.49
Full-time 2012.29
24
Input data
Output after
applying delete
command
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-13
Substitute Command (s)
Syntax:
[addr1][,addr2] s/search/replace/[flags]
replaces text selected by search string with replacement string
search string can be regular expression
flags:
◦ global (g), i.e. replace all occurrences
◦ specific substitution count (integer), default 1
25
TOPIC 6 – SED – STREAM EDITOR
Substitution Back References
27
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-14
Example: Replacement String &
28
$ cat datafile
Charles Main 3.0 .98 3 34
Sharon Gray 5.3 .97 5 23
Patricia Hemenway 4.0 .7 4 17
TB Savage 4.4 .84 5 20
AM Main Jr. 5.1 .94 3 13
Margot Weber 4.5 .89 5 9
Ann Stephens 5.7 .94 5 13
$ sed -e ‘s/[0-9][0-9]$/&.5/’ datafile
Charles Main 3.0 .98 3 34.5
Sharon Gray 5.3 .97 5 23.5
Patricia Hemenway 4.0 .7 4 17.5
TB Savage 4.4 .84 5 20.5
AM Main Jr. 5.1 .94 3 13.5
Margot Weber 4.5 .89 5 9
Ann Stephens 5.7 .94 5 13.5
TOPIC 6 – SED – STREAM EDITOR
Example: Back Reference
$ cat filedata
/home/ux/user/z156256
/home/ux/user/z056254
/home/lx/user/z106253
/home/ux/user/z150252
/home/mp/user/z056254
/home/lx/user/z106253
$ sed -e ‘s,/home/\(..\)/user/\(z[0-9]\{6\}\),/usr/\2/\1,g’ filedata
/usr/z156256/ux
/usr/z056254/ux
/usr/z106253/lx
/usr/z150252/ux
/usr/z056254/mp
/usr/z106253/lx 29
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-15
sed i/o commands
31
TOPIC 6 – SED – STREAM EDITOR
Input (next) Command: n and N
Forces sed to read the next input line
◦ Copies the contents of the pattern space to output
◦ Deletes the current line in the pattern space
◦ Refills it with the next input line
◦ Continue processing
N (uppercase) Command
◦ adds the next input line to the current contents of the pattern space
◦ useful when applying patterns to two or more lines at the same time
32
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-16
Output Command: p and P
Print Command (p)
◦ copies the entire contents of the pattern space to output
◦ will print same line twice unless the option “–n” is used
Print command: P
◦ prints only the first line of the pattern space
◦ prints the contents of the pattern space up to and including a new line
character
◦ any text following the first new line is not printed
33
TOPIC 6 – SED – STREAM EDITOR
File commands
allows to read and write from/to file while processing standard input
read: r command
write: w command
42
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-17
Read File command
Syntax: r filename
◦ queue the contents of filename to be read and inserted into the output
stream at the end of the current cycle, or when the next input line is read
◦ if filename cannot be read, it is treated as if it were an empty file, without
any error indication
◦ single address only
43
TOPIC 6 – SED – STREAM EDITOR
Write File command
Syntax: w filename
◦ Write the pattern space to filename
◦ The filename will be created (or truncated) before the first input line is read
◦ all w commands which refer to the same filename are output through the
same FILE stream
44
TOPIC 6 – SED – STREAM EDITOR
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-18
Example: The quit (q) Command
Syntax: [addr]q
◦ Quit (exit sed) when addr is encountered.
Example: Display the first 50 lines and quit
% sed -e ’50q’ datafile
Same as:
% sed -n -e ‘1,50p’ datafile
% head -50 datafile
47
TOPIC 6 – SED – STREAM EDITOR