CS计算机代考程序代写 python javascript database Java gui Working with Files and Processes

Working with Files and Processes
IFN507 Lecture 10

Outline

Text Editing

Searching and Manipulating Text

Displaying the Contents of a File

Process Management

Redirections and Pipes

Text Editing

Learning to use text editors, why?
Used in programming/scripting, editing configuration/text files
Text editors
Command line, e.g.
Vi (vim)
Nano
Emacs
GUI text ediors
Emacs (graphical version)
Gedit
Endless other choices some free, some paid
4
学习使用文本编辑器，为什么?

用于编程/脚本，编辑配置/文本文件

文本编辑器

命令行。

Vi (vim)

纳米

Emacs

GUI文本ediors

Emacs(图形版本)

中用户

无尽的选择有些是免费的，有些是付费的

The vi Editor
5

One of the oldest and most popular text editors available for UNIX OSs.

Linux equivalent of vi: vim (vi improved)

Standard on most Linux distributions

vi editor is not the easier of the editors to use, but it has the advantage of portability

vi editor are nearly identical across Linux and UNIX systems.

vi supports regular expressions and can perform > 1000 different functions for the user

To open an existing Text file for editing, type vi filename (or vim filename)

To open a new file for editing, type vi or vim at the command line.

The vi Editor – cont.

vi has two modes:
Command mode: performs text editing tasks not related to inserting text, e.g. deleting text, copying text, saving changes to a file, and exiting the vi editor
Insert mode: inserts text, but nothing else.
Press the Esc key to return to command mode
Start in command mode
Insertion mode by entering an insertion (i key)

6
命令模式:执行与文本插入无关的文本编辑任务，如删除文本、复制文本、保存文件更改、退出vi编辑器等

插入模式:插入文本，但不插入其他内容。

按Esc键返回命令模式

在命令模式下启动

通过输入插入(i键)进入插入模式

Text Editor – Emacs

GNU Emacs (Editor MACroS) editor: offers comparable functionality to vi
$ sudo apt install emacs

7
GNU Emacs(编辑器MACroS)编辑器:提供与vi相当的功能

$ sudo apt安装emacs

Text Editor

Nano editor:
text editor that uses Ctrl key combinations for performing functions

纳米编辑器:文本编辑器，使用Ctrl键组合来执行功能

Text Editors – gedit

The gedit editor: graphical text editor functional in a GUI environment
Access gedit functionality from the drop-down menu.
编辑器:图形文本编辑器，在GUI环境中起作用

从下拉菜单访问gedit功能

Display the Contents of a File
10

Useful Commands
11

cat

tac

head

tail

less

most

cat

Concatenate: view an entire text file on the terminal screen
The cat command: displays contents of a text file to the screen
The -n option: displays line number of each line in the file
译文：
连接:在终端屏幕上查看整个文本文件

cat命令:将文本文件的内容显示到屏幕上

-n选项:显示文件中每一行的行号

tac

Log files: contain records of events
Most recent events are appended to end of the file
The tac command displays contents of a text file in reverse order
日志文件:包含事件记录

大多数最近的事件被追加到文件的末尾

tac命令以相反的顺序显示文本文件的内容

head and tail

Head command: displays the first 10 lines of a file
head test1
head -3 test1
Tail command: displays the last 10 lines of a file
tail test1
tail -3 test1

14
Head命令:显示文件头test1 Head -3 test1的前10行

Tail命令:显示文件Tail test1 Tail -3 test1的最后10行

Large text files can not be viewed using the cat command
Screen will only fit a portion of the file
The more command: displays text files page-by-page
The less command: same as more command, but can also use arrow keys
The more and less commands can be used with output of other commands

15
使用cat命令无法查看较大的文本文件

屏幕将只适合文件的一部分

more命令:逐页显示文本文件

少命令:与多命令相同，但也可以使用方向键

more和less命令可以与其他命令的输出一起使用

less

Less is a used for displaying text files.
Less is not an editor, cannot used for changing the contents of the file
Less allows searching forwards and backwards
Less supports pattern search

16
Less用于显示文本文件。

Less不是一个编辑器，不能用于更改文件的内容

Less允许向前和向后搜索

Less支持模式搜索

most

most has more functions than more and less.
most supports opening multiple files at the same time.
For longer rows, most does not truncate them into multi-line displays, but provides the ability to scroll left and right to display in the same line.

17
大多数都有更多的功能而不是更多和更少。大多数支持同时打开多个文件。

对于较长的行，大多数不会将其截断为多行显示，而是提供向左和向右滚动以在同一行中显示的能力。

Redirections and Pipes
This Photo by Unknown Author is licensed under CC BY-SA
18

STDOUT

STDOUT (Standard Output)
Linux treats every object as a file including output processes like displaying text on the screen. Each of these objects is identified using a file descriptor integer.
The file descriptor that identifies output from a command/script is 1. This is also called STDOUT.
The default behaviour when executing Linux terminal commands is to output the result to STDOUT
STDOUT by default directs its output to the current terminal screen (/dev/tty)
You can redirect this output

STDOUT(标准输出)

Linux将每个对象视为一个文件，包括在屏幕上显示文本等输出过程。每个对象都使用文件描述符整数标识。

标识命令/脚本输出的文件描述符为1。这也称为STDOUT。

执行Linux终端命令时的默认行为是将结果输出到STDOUT

STDOUT默认将其输出定向到当前终端屏幕(/dev/tty)

您可以重定向此输出
19

STDOUT continued…
The echo command takes the user provided arguments and outputs it to STDOUT.

STDOUT continued…

You can use redirection operators to redirect this output to other files
>
Redirects the output of a command to a file
>>
Redirects and appends the output of a command to a file

21
您可以使用重定向操作符将此输出重定向到其他文件

将命令的输出重定向到文件

将命令的输出重定向并追加到文件中

STDERR

STDERR (Standard Error)
When a script file or command error occurs, its file descriptor identifier is 2, also identified as STDERR.
STDERR by default outputs to the terminal much like STDOUT.

STDERR(标准误差)

当发生脚本文件或命令错误时，其文件描述符标识符为2，也标识为STDERR。

STDERR在默认情况下输出到终端，很像STDOUT。
22

STDERR continued…

Like STDOUT, you can redirect the output of STDERR to other files
2>
Redirects error messages from a command or script to a file
2>>
Redirects and appends error messages from a command or script to a file

23
与STDOUT类似，您可以将STDERR的输出重定向到其他文件

2 >

将错误消息从命令或脚本重定向到文件

2 > >

将错误消息从命令或脚本重定向并追加到文件

STDIN

STDIN (Standard Input)
Usually input enters the system through the keyboard. Input has the file descriptor 0 and is represented by STDIN.
The redirection of STDIN Not as common as STDOUT and STDERR, however some commands take input from STDIN

STDIN(标准输入)

输入通常通过键盘输入系统。输入的文件描述符为0，由STDIN表示。

STDIN的重定向不像STDOUT和STDERR那么常见，但是有些命令接受来自STDIN的输入
24

STDIN continued…

< The redirection operator for STDIN STDIN的重定向操作符 25 Redirection Operators Summary Operator Description > Redirects STDOUT to a file. Overwrites the contents of the file if it already exists.
>> Appends STDOUT to a file. If the file doesn’t exist it will create it.
2> Redirects STDERR to a file. Overwrites the contents of the file if it already exists.
2>> Appends STDERR to a file. If the file doesn’t exist it will create it.
&> Redirects both STDOUT and STDERR to a file. Overwrites the contents of the file if it already exists.
&>> Appends both STDOUT and STDERR to a file. If the file doesn’t exist it will create it.
< Redirects STDIN from a file to a command <> Redirects STDIN from a file to a command and redirects STDOUT to the file

Piping

27
管道提供了在多个命令之间重定向STDOUT、STDIN和STDERR的能力
管道操作符是竖条字符|
可以用于从一个命令连接STDOUT到另一个命令的STDIN (command1 | command2 | command 3…)

Piping provides the ability to redirect STDOUT, STDIN and STDERR between multiple commands

The pipe operator is the vertical bar character |

Can be used to connect the STDOUT from one command to the STDIN of another (command1 | command2 | command 3 …)

Piping continued…

Searching and Manipulating Text

Regular Expressions (regex)

grep

sed

Searching for Text Within Files

Text tools
Commands that search for and manipulate text
Such as database information
Regular expressions (regexp)
Text wildcards that ease the search for specific text
Recognized by several text tools and programming languages
Grep
Awk
Sed
Vi
Emasc
Perl
python

31
文本工具

搜索和操作文本的命令

例如数据库信息

正则表达式(正则表达式)

简化特定文本搜索的文本通配符

可被多种文本工具和编程语言识别

Grep

Awk

Sed

Emasc

Perl python

Regular Expressions (regex)

Different from wildcard metacharacters
Wildcard metacharacters are interpreted by the shell, whereas regular expressions are interpreted by a text tool program
Wildcard metacharacters match characters in filenames (or directory names) on a Linux filesystem, whereas regular expressions match characters within text files on a Linux filesystem
Wildcard metacharacters typically have different definitions than regular expression metacharacters
More regular expression metacharacters are available than wildcard metacharacters

32
不同于通配符元字符
通配符元字符由shell解释，而正则表达式由文本工具程序解释
通配符元字符在Linux文件系统中匹配文件名(或目录名)中的字符，而正则表达式在Linux文件系统中匹配文本文件中的字符
通配符元字符通常与正则表达式元字符有不同的定义
正则表达式元字符比通配符元字符可用

Regular Expression (regex)

A sequence of characters that specifies a search pattern used for matching one or more characters within a string.
Matching patterns can be specific characters, wildcards and ranges of characters
Used in
Data validation
transform data format
String parsing
String replacement, e.g. “find” or “find and replace” operations
33
指定用于匹配字符串中的一个或多个字符的搜索模式的字符序列。

匹配模式可以是特定字符、通配符和字符范围

中使用的

数据验证转换数据格式

字符串解析

字符串替换,如。“查找”或“查找并替换”操作

Regular Expression (regex) – cont.

Formal language for describing a set of strings.
Find, extract, replace, delete, add parts of strings
Log analysis, file parsing, report generation …
Many programming languages contains a regex parser

Supported in
Scripting languages, e.g. Perl, Python, PHP, and JavaScript
General purpose programming languages, e.g. Java;
Word processors, e.g. Word for searching texts
Text editors, vi, grep

34
描述一组字符串的形式语言。

查找、提取、替换、删除、添加字符串的部分

日志分析，文件解析，报表生成…

许多编程语言都包含正则表达式解析器

支持

脚本语言，例如Perl、Python、PHP和JavaScript

通用编程语言，如Java;

文字处理器，例如用于搜索文本的Word

文本编辑器，vi, grep

Regular Expressions – cont.
Regular expressions are divided into two different categories
Common regular expressions: available to most text tools
Extended regular expressions: less common and available in only certain text tools
Regular expressions are quite complex, only some basic regular expressions will be covered in this unit
We will use POSIX Basic Regular Expression Syntax in this unit
https://en.wikibooks.org/wiki/Regular_Expressions/POSIX_Basic_Regular_Expressions
35
正则表达式分为两类

通用正则表达式:大多数文本工具都可以使用

扩展正则表达式:较不常见，仅在某些文本工具中可用

正则表达式非常复杂，本单元只介绍一些基本的正则表达式

我们将在这个单元https://en.wikibooks.org/wiki/Regular_Expressions/POSIX_Basic_Regular_Expressions中使用POSIX基本正则表达式语法

Regular expression definitions and examples

Regex Description Example Type
* Matches 0 or more occurrences of the previous character better* matches better, betterr, betterrrr, betterrrrr, and so on Common
. (period) Matches any single character, except line break characters better. matches bettera, better, better, better1, better2, better3, and so on. Common
[…] Matches one character from the range specified within the braces better[1238] matches better1, better2, better3, and better8;
better[a-c] matches bettera, betterb, and better Common
[^…] Matches one character NOT from the range specified within the braces better[^1238] matches better4, better5, better6 and better7, bettera, better, so on (any character except 1, 2, 3, or 8) Common
^ Matches the following characters if they are the first characters on the line ^better matches better if better is the first set of the character in the line Common
$ Matches the previous characters if they are the last characters on the line better$ matches better if better is the last set of characters in the line Common
? Matches 0 or 1 occurrences of the previous character better? matches bette, better Extended
(… | …) Matches either of two sets of characters (dog|cat) matches the word “dog” or “cat” Extended

Regular Expressions – cont.

Regular expressions are divided into two different categories
Common regular expressions: available to most text tools
Extended regular expressions: less common and available in only certain text tools

正则表达式分为两类

通用正则表达式:大多数文本工具都可以使用

扩展正则表达式:较不常见，仅在某些文本工具中可用

Regex Description Example Type
* Matches 0 or more occurrences of the previous character letter* matches lette, letter, letterr, letterrrr, letterrrrr Common
? Matches 0 or 1 occurrences of the previous character letter? matches lette, letter Extended
[…] Matches one character from the range specified within the braces letter[1238] matches letter1, letter2, letter3, and letter8; letter[a-c] matches lettera, letterb, and letter common
(… | …) Matches either of two sets of characters (mother|father) matches the word “mother” or “father” Extended
… … … …

The grep Command

The global regular expression print (grep) command: displays lines in a text file that match common regular expressions
The egrep command: displays lines in a text file that match extended regular expressions
The fgrep command: does not interpret any regular expressions and returns results much faster

38
全局正则表达式打印(grep)命令:显示文本文件中与普通正则表达式匹配的行

egrep命令:显示文本文件中匹配扩展正则表达式的行

fgrep命令:不解释任何正则表达式，返回结果要快得多

The grep Command

Requires two arguments
Text to search for
Files to search
Case sensitive
For case-insensitive search, use –i option
Matches patterns of text
Unable to discern words or phrases unless they are specified

Introduction sed

A powerful text stream editor
Performs text processing functions on a file for searching, replacement, insertion or deletion.
Most common use of the SED command is substitution.
can edit files even without opening it.
Supports regular expression which allows it perform complex pattern matching.

How sed works

Sed receives its input from standard input, changes that input as directed by commands in a command file, and then writes the resulting stream to standard output.
If you do not provide a command file and do not use any flags with the sed command, sed copies standard input to standard output without change.
Input sources
Input stream: either from one or more files or entered directly form the keyboard
Commands: a set of addresses and associated commands to be performed
[Line1 [,Line2] ] command [argument]
The parameters Line1 and Line2 are called addresses. Addresses can be either patterns to match in the input stream, or line numbers in the input stream.

Sed syntax

sed s/pattern/replacement/[flags]
s: substitute
flags n,g,p,w
n: replace nth instance of pattern with replacement
g: replace all instances of pattern with replacement
p: write pattern space to STDOUT if successful
w file: write the pattern space to file if successful

Sed – substituting string

sed ‘s/My/Her/’ mydata100
To substitute My to Her
The “s” specifies the substitution operation
The “/” are delimiters
The “My” is the search pattern
and the “Her” is the replacement string
By default, the sed command replaces the first occurrence of the pattern in each line and it will not replace the second, third…occurrence in the line.
sed ‘s/My/Her/2’ mydata100
Replacing the nth occurrence of a pattern in a line : Use the /1, /2 etc flags to replace the first, second occurrence of a pattern in a line

I am as quick as a cheetah
My eyes are as blue as the ocean
My ears can hear like a rabbit
My brain is as smart as a scientist
My nose can smell like a bear
My heart is as big as an elephant
My my my …

File name: mydata100
43

Sed – substituting string – cont.
sed ‘s/My/Her/g’ mydata100
Replacing all the occurrence of the pattern in a line :
The substitute flag /g (global replacement) specifies the sed command to replace all the occurrences of the string in the line.
sed ‘s/My/Her/’ < mydata100 > mydata200
Searching a pattern with an input from a file and replacing with a pattern and write the result to another file.

Sed – substituting string – cont.
echo “This is a brown dog” | sed ’s/brown/green/’
sed can read from STDIN

echo “This is a brown dog” | sed -e ’s/brown/green/’ -e ‘s/dog/cat/’ data1_file
-e provides a way of separating the commands

Extract names from /etc/passwd file
sed ‘s/$[^:]*$.*/ \1/’ /etc/passwd

substitution
delimiter
delimiter
delimiter
Searching patterns
46

‘s/ the sed command and the beginning of the substitution expression

\( the opening parenthesis, preceded by a backslash \

[^:]* The first subexpression of the search term contains a group in square brackets. The caret ^ means “not” when used in a group.

\) The closing parenthesis with a preceding backslash \

.* This 2nd search subexpression means “any character and any number of them.”

/\1 The substitution portion of the expression contains 1 preceded by a backslash \. This represents the text that matches the first subexpression.

/’ The closing forward-slash / and single quote ‘ terminate the sed command.

Reverse sub-strings in a specific order
Searching patterns
substitution
echo “Lily, Kane” | sed ‘s/$.*$, $.*$/\2,\1 /g’

delimiter
First name
Last name
Pipe
delimiter
delimiter
47

pipe lets you sends the output of one command to another

sed ‘s/ the substitution command.

$.*$, The 1st subexpression is any number of any characters

$.*$ The next subexpression is any number of any characters

\2,\1 The two subexpressions in parentheses, we can refer to both of them by their numbers. As we want to reverse the order, we type them as second-match, first-match. The numbers have to be preceded by a backslash (\).

/g: to work globally on each line.

Sed – deleting string
sed command can be used for deleting lines from a particular file.
sed command is used for performing deletion operation without even opening the file
sed ‘nd’ filename
sed ‘7d’ mydata101

Sed – deleting string – cont.
49

To delete a last line

sed ‘$d” filename

To delete line from range x to y

sed 1,3d filename

To delete from 2nd to last line

sed ‘2,$d’ filename

To delete pattern matching line

sed ‘/pattern/d ‘ filename

Example: sed ‘/cheetah/d’ filename

To delete empty lines or blank lines

sed ‘/^$/d’ filename

To delete lines other than the first line or header line

sed ‘1!d’ filename

Process Management
This Photo by Unknown Author is licensed under CC BY-SA
50

Linux Processes

Program: executable file
Can be run when executed
Process: program running in memory and on CPU
A program in action
User process: process run on a terminal
Daemon process: system process
Not associated with a terminal
51

Linux processes – cont.
Process #100
PID = 100000
PPID = 10000
Process #101
PID = 100100
PPID = 100000
Process #102
PID = 100200
PPID = 100000
Process #103
PID = 100300
PPID = 100100
Process #104
PID = 100400
PPID = 100100
52

Process ID (PID):

unique identifier assigned to a process

Child process:

process started by another process (parent process)

Parent process:

process that has started other processes (child processes)

Parent Process ID (PPID):

identifies the process that started it

Linux processes – cont.

The init daemon starts most other daemons during the system initialization process
Including those that allow for user logins
The login program starts a BASH shell
BASH shell then interprets user commands and starts all user processes
Each process on the Linux system can be traced back to the init daemon by examining PPIDs

Init daemon
daemon
daemon
daemon
User login
BASH shell
User
process
User
process
User
process
53

Viewing Processes

The ps command: used to view processes
Most versatile and common process viewing utility
The –f (full) option: more complete information
User identifier (UID), PPID, start time, CPU utilization
The -e option: displays the entire list of processes across all terminals including daemons

Viewing Processes – cont.
Process flag: indicates particular features of the process
Process state: current processor state of process
Sleeping (S) or running (R)
Zombie process: process is finished, but parent has not released child process’s PID
Zombie processes are also known as defunct process
Process state is Z
To view a list of zombie processes on your system, use ps -el

Viewing Processes – cont.

Process priority (PRI): priority used by the kernel for the process
Measured between 0 (high priority) and 127 (low priority)
Nice value (NI): can be used to affect the process priority indirectly
Measured between -20 (a greater chance of a high priority) and 19 (a greater chance of a lower priority)
The size of the process in memory (SZ) is listed and measured in kilobytes

Viewing Processes – cont.

Some options to the ps command are not prefixed by a dash (-) character
Referred to as Berkeley style options
Two most common of these are:
The a option: lists all processes across terminals
The x option: lists processes that do not run on a terminal
The pstree command: displays the lineage of a process by tracing its PPIDs until the init daemon

Viewing Processes – cont.

Common options to the ps command
Option Description
-e Displays all processes running on terminals as well as processes that do not run on a terminal (daemons)
-f Displays a full list of information about each process, including the UID, PID, PPID, CPU utilization, start time, terminal, processor time, and command name
-l Displays a long list of information about each process, including the flag, state, UID, PID, PPID, CPU utilization, priority, nice value, address, size, WCHAN, terminal, and command name
-Z Displays security-related information about each process
A Displays all processes running on terminals
x Displays all processes that do not run on terminals

Viewing Processes – cont.

The top command: displays an interactive screen listing processes
Organized by processor time
Processes using most processor time are listed first
Rogue process: faulty process
Consumes excessive system resources
The top command can be used to change the priority of processes or to kill processes
Rogue processes can be killed immediately when identified

Killing Processes

The kill command: sends a kill signal to a process to terminate it
64 types of kill signals
Each affects processes in different ways
-l option: displays a list of kill signal names and associated numbers

Killing Processes – cont.

Source from Table 9-2
Eckert, Linux+ and LPIC-1 Guide to Linux Certification, 5th Edition. Cengage
Name Number Description
SIGHUP 1 Also known as the hang-up signal, it stops a process, then restarts it with the same PID. If you edit the configuration file used by a running daemon, that daemon might be sent a SIGHUP to restart the process; when the daemon starts again, it reads the new configuration file.
SIGINT 2 This signal sends an interrupt signal to a process. Although this signal is one of the weakest kill signals, it works most of the time. When you use the Ctrl+c key combination to kill a currently running process, a SIGINT is actually being sent to the process.
SIGQUIT 3 Also known as a core dump, the quit signal terminates a process by taking the process information in memory and saving it to a file called core on the hard disk in the current working directory. You can use the Ctrl+\ key combination to send a SIGQUIT to a process that is currently running.
SIGTERM 15 The software termination signal is the most common kill signal used by programs to kill other processes. It is the default kill signal used by the kill command.
SIGKILL 9 Also known as the absolute kill signal, it forces the Linux kernel to stop executing the process by sending the process’s resources to a special device file called /dev/null.

Killing Processes – cont.

To send a kill signal to a process, specify the kill signal to send as an option to the kill command, followed by the appropriate PID of the process.
For example, to send a SIGKILL to a process called nano, you could use the these commands to locate and terminate the process.

Killing Processes – cont.

Trap: the ability to ignore a kill signal
The SIGKILL signal cannot be trapped by any process
Use only as last resort because it prevents a process from closing temporary files and resources properly
When a parent process receives a kill signal, the parent process terminates all child processes before terminating itself
To kill several related processes, send signal to parent process
Kill parent process in order to kill zombie processes

Killing Processes – cont.
The killall command: kills multiple processes of the same name in one command
Takes kill signal number as an option
Uses process name instead of PID
The pkill command: kill processes by process name
Allows you to identify process names using regular expressions as well as specify other criteria

Process priorities

Time slice: amount of time a process is given on a CPU
More time slices mean more execution time on CPU
Executes faster
Usually measured in milliseconds

Process priorities – cont.
PRI dictates number of time slices a process gets
Cannot change PRI value directly
Set the nice value (NI) to indirectly affect priority
A negative NI value, more time slices; a positive NI value, less time slices
Processes start with NI of 0 by default
The nice command: change a process’s priority as it starts
The renice command: alter process priority after it has been started
Root user can use the renice command to change the priority of all processes that are owned by a certain user or group

References

Jason W. Eckert, Linux+ and LPIC-1 Guide to Linux Certification, 5th Edition, Cengage
Christine Bresnahan, Richard Blum, LPIC-1 Linux Professional Institute Certification Study Guide, 5th Edition, Sybex
67

End of Lecture

/docProps/thumbnail.jpeg

Related Posts