Topic6
SED – STREAM EDITOR
CSCI 330 – The UNIX System
What is sed?
A non-interactive stream editor
Interprets sed instructions and performs actions
Use sed to:
◦ Automatically perform edits on file(s)
◦ Simplify doing the same edits on multiple files ◦ Write conversion programs
NIU – Department of Computer Science
11-1
TOPIC 6 – SED – STREAM EDITOR
2
The sed command
CSCI 330 – The UNIX System
sed command syntax
TOPIC 6 – SED – STREAM EDITOR
3
NIU – Department of Computer Science
11-2
TOPIC 6 – SED – STREAM EDITOR
4
sed Operation
CSCI 330 – The UNIX System
How Does sed Work?
sed reads line of input
◦ line of input is copied into a temporary buffer called pattern space ◦ editing commands are applied
◦ subsequent commands are applied to line in the pattern space, not the original input line
◦ once finished, line is sent to output (unless –n option was used)
◦ line is removed from pattern space
sed reads next line of input, until end of file
Note: input file is unchanged
5
NIU – Department of Computer Science
11-3
TOPIC 6 – SED – STREAM EDITOR
6
TOPIC 6 – SED – STREAM EDITOR
sed instruction format
address determines which lines in the input file are to be processed by the command(s)
◦ if no address is specified, then the command is applied to each input line
address types:
◦ Single-Line address
◦ Set-of-Lines address
◦ Range address
◦ Nested address 7
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-4
Single-Line Address
Specifies only one line in the input file
◦ special: dollar sign ($) denotes last line of input file
Examples:
◦ show only line 3
sed -n -e ‘3 p’ input-file
◦ show only last line
sed -n -e ‘$ p’ input-file
◦ substitute “endif” with “fi” on line 10
sed -e ’10 s/endif/fi/’ input-file
TOPIC 6 – SED – STREAM EDITOR
8
TOPIC 6 – SED – STREAM EDITOR
Set-of-Lines Address
use regular expression to match lines ◦ written between two slashes
◦ process only lines that match
◦ may match several lines
◦ lines may or may not be consecutives
Examples:
sed -e ‘/key/ s/more/other/’ input-file sed -n -e ‘/r..t/ p’ input-file
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-5
Range Address
Defines a set of consecutive lines Format:
start-addr,end-addr (inclusive)
Examples: 10,50
10,/R.E/ /R.E./,10 /R.E./,/R.E/
line-number,line-number line-number,/RegExp/
/RegExp/,line-number /RegExp/,/RegExp/
TOPIC 6 – SED – STREAM EDITOR
9
10
TOPIC 6 – SED – STREAM EDITOR
Example: Range Address
% sed -n -e ‘/^BEGIN$/,/^END$/p’ input-file
addr1 addr2
Print lines between BEGIN and END, inclusive
CSCI 330 – The UNIX System
BEGIN
Line 1 of input
Line 2 of input
Line3 of input
END
Line 4 of input
Line 5 of input
These lines are printed
Nested Address
11
TOPIC 6 – SED – STREAM EDITOR
Nested address contained within another address
Example:
print blank lines between line 20 and 30
20,30{ /^$/ p
}
NIU – Department of Computer Science
11-6
TOPIC 6 – SED – STREAM EDITOR
12
Address with !
address with an exclamation point (!):
instruction will be applied to all lines that do not match the address
Example:
print lines that do not contain “obsolete”
sed -e ‘/obsolete/!p’ input-file
CSCI 330 – The UNIX System
sed commands
TOPIC 6 – SED – STREAM EDITOR
13
NIU – Department of Computer Science
11-7
14
Line Number
line number command (=) writes the current line number before each matched/output line
Examples:
sed -e ‘/Two-thirds-time/=’ tuition.data sed -e ‘/^[0-9][0-9]/=’ inventory
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-8
modify commands
TOPIC 6 – SED – STREAM EDITOR
Substitute
15
16
TOPIC 6 – SED – STREAM EDITOR
Insert Command: i
adds one or more lines directly to the output before the address:
◦ inserted “text” never appears in sed’s pattern space
◦ cannot be used with a range address; can only be used with the single-line and set-of-lines address types
Syntax:
[address] i\
text
TOPIC 6 – SED – STREAM EDITOR
Example: Insert Command (i)
% cat tuition.insert.sed 1 i\
Tuition List\
Sed script to insert “Tuition List” as report title before line 1
CSCI 330 – The UNIX System
17
% cat tuition.data
Part-time 1003.99
Two-thirds-time 1506.49
Full-time 2012.29
% sed -f tuition.insert.sed tuition.data
Tuition List
Input data
Output after applying the insert command
NIU – Department of Computer Science
11-9
Part-time Two-thirds-time Full-time
1003.99 1506.49 2012.29
TOPIC 6 – SED – STREAM EDITOR
18
Append Command: a
adds one or more lines directly to the output after the address:
◦ Similar to the insert command (i), append cannot be used with a range address.
◦ Appended “text” does not appear in sed’s pattern space.
Syntax:
[address] a\
text
TOPIC 6 – SED – STREAM EDITOR
Example: Append Command
(a)
% cat tuition.append.sed
a\
————————–
% cat tuition.data
Part-time 1003.99
Two-thirds-time 1506.49
Full-time 2012.29
% sed -f tuition.append.sed tuition.data Part-time 1003.99 ————————– Two-thirds-time 1506.49 ————————–
Full-time 2012.29 ————————–
Sed script to append dashed line after each input line
Input data
Output after applying the append command
CSCI 330 – The UNIX System
19
NIU – Department of Computer Science
11-10
TOPIC 6 – SED – STREAM EDITOR
20
Change Command: c
replaces an entire matched line with new text accepts four address types:
◦ single-line, set-of-line, range, and nested addresses.
Syntax:
[address1[,address2]] c\
text
NIU – Department of Computer Science
11-11
Example: Change Command (c)
% cat tuition.change.sed
1 c\ tuition cost from
Part-time 1100.00
% cat tuition.data
Part-time 1003.99
Two-thirds-time 1506.49
Full-time 2012.29
% sed -f tuition.change.sed tuition.data
Part-time 1100.00
Two-thirds-time 1506.49
Output after applying the change command
TOPIC 6 – SED – STREAM EDITOR
Sed script to change 1003.99 to 1100.00
Input data
CSCI 330 – The UNIX System
21
Full-time
2012.29
22
TOPIC 6 – SED – STREAM EDITOR
Delete Command: d
deletes the entire pattern space
◦ commands following the delete command are ignored since the deleted text is no longer in the pattern space
CSCI 330 – The UNIX System
Syntax:
[address1[,address2]] d
TOPIC 6 – SED – STREAM EDITOR
Example: Delete Command (d)
Remove part-time data from “tuition.data” file
% cat tuition.data
Part-time 1003.99
Two-thirds-time 1506.49 Input data Full-time 2012.29
% sed –e ‘/^Part-time/d’ tuition.data
23
NIU – Department of Computer Science
11-12
Two-thirds-time 1506.49
Output after applying delete command
Full-time
2012.29
TOPIC 6 – SED – STREAM EDITOR
24
Substitute Command (s)
Syntax:
[addr1][,addr2] s/search/replace/[flags]
replaces text selected by search string with replacement string
search string can be regular expression
flags:
◦ global (g), i.e. replace all occurrences
◦ specific substitution count (integer), default 1
CSCI 330 – The UNIX System
TOPIC 6 – SED – STREAM EDITOR
Substitution Back References
25
NIU – Department of Computer Science
11-13
TOPIC 6 – SED – STREAM EDITOR
27
Example: Replacement String &
$ cat datafile
Charles Main
Sharon Gray
Patricia Hemenway
TB Savage
AM Main Jr.
Margot Weber
Ann Stephens
3.0 .98 3 34 5.3 .97 5 23 4.0 .7 4 17 4.4 .84 5 20 5.1 .94 3 13 4.5 .89 5 9 5.7 .94 5 13
$ sed -e ‘s/[0-9][0-9]$/&.5/’ datafile
Charles Main
Sharon Gray
Patricia Hemenway
TB Savage
AM Main Jr.
Margot Weber
Ann Stephens
3.0 .98 3 34.5
5.3 .97 5 23.5
4.0 .7 4 17.5
4.4 .84 5 20.5
5.1 .94 3 13.5
9 28 5.7 .94 5 13.5
4.5 .89 5
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-14
TOPIC 6 – SED – STREAM EDITOR
Example: Back Reference
$ cat filedata /home/ux/user/z156256 /home/ux/user/z056254 /home/lx/user/z106253 /home/ux/user/z150252 /home/mp/user/z056254 /home/lx/user/z106253
$ sed -e ‘s,/home/\(..\)/user/\(z[0-9]\{6\}\),/usr/\2/\1,g’ filedata /usr/z156256/ux
/usr/z056254/ux
/usr/z106253/lx
/usr/z150252/ux /usr/z056254/mp /usr/z106253/lx
29
TOPIC 6 – SED – STREAM EDITOR
sed i/o commands
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-15
TOPIC 6 – SED – STREAM EDITOR
Input (next) Command: n and N
Forces sed to read the next input line
◦ Copies the contents of the pattern space to output ◦ Deletes the current line in the pattern space
◦ Refills it with the next input line
◦ Continue processing
N (uppercase) Command
◦ adds the next input line to the current contents of the pattern space ◦ useful when applying patterns to two or more lines at the same time
31
32
TOPIC 6 – SED – STREAM EDITOR
Output Command: p and P
Print Command (p)
◦ copies the entire contents of the pattern space to output ◦ will print same line twice unless the option “–n” is used
Print command: P
◦ prints only the first line of the pattern space
◦ prints the contents of the pattern space up to and including a new line character
◦ any text following the first new line is not printed
CSCI 330 – The UNIX System
NIU – Department of Computer Science
11-16
TOPIC 6 – SED – STREAM EDITOR
File commands
allows to read and write from/to file while processing standard input read: r command
write: w command
33
42
TOPIC 6 – SED – STREAM EDITOR
Read File command
Syntax: r filename
◦ queue the contents of filename to be read and inserted into the output
stream at the end of the current cycle, or when the next input line is read
◦ if filename cannot be read, it is treated as if it were an empty file, without any error indication
◦ single address only
CSCI 330 – The UNIX System
TOPIC 6 – SED – STREAM EDITOR
Write File command
Syntax: w filename
◦ Write the pattern space to filename
◦ The filename will be created (or truncated) before the first input line is read
◦ all w commands which refer to the same filename are output through the same FILE stream
43
NIU – Department of Computer Science
11-17
TOPIC 6 – SED – STREAM EDITOR
44
Example: The quit (q) Command
CSCI 330 – The UNIX System
Syntax: [addr]q
◦ Quit (exit sed) when addr is encountered.
Example: Display the first 50 lines and quit
% sed -e ’50q’ datafile
Same as:
% sed -n -e ‘1,50p’ datafile
% head -50 datafile
NIU – Department of Computer Science
11-18
TOPIC 6 – SED – STREAM EDITOR
47