CS计算机代考程序代写 database data structure cache 2021/4/28 PostgreSQL File Manager

程序代写 CS代考 / data structure, database

2021/4/28 PostgreSQL File Manager
PostgreSQL File Manager
PostgreSQL File Manager Relations as Files
File Descriptor Pool
File Manager
>>
COMP9315 21T1 ♢ PG File Manager ♢ [0/15]
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
1/17

2021/4/28 PostgreSQL File Manager
❖ PostgreSQL File Manager PostgreSQL uses the following le organisation …
∧ >>
COMP9315 21T1 ♢ PG File Manager ♢ [1/15]
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
2/17

2021/4/28 PostgreSQL File Manager
❖ PostgreSQL File Manager (cont) Components of storage subsystem:
mappingfromrelationstoles (RelFileNode) abstractionforopenrelationpool (storage/smgr) functionsformanagingles (storage/smgr/md.c) le-descriptorpool (storage/file)
PostgreSQL has two basic kinds of les: heap les containing data (tuples) index les containing index entries
Note: smgr designed for many storage devices; only disk handler provided
COMP9315 21T1 ♢ PG File Manager ♢ [2/15]
<< ∧ >>
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
3/17

2021/4/28 PostgreSQL File Manager
❖ Relations as Files
PostgreSQL identies relation les via their OIDs. The core data structure for this is RelFileNode:
typedef struct RelFileNode { Oid spcNode; // tablespace Oid dbNode; // database Oid relNode; // relation
} RelFileNode;
Global (shared) tables (e.g. pg_database) have
spcNode == GLOBALTABLESPACE_OID dbNode == 0
COMP9315 21T1 ♢ PG File Manager ♢ [3/15]
<< ∧ >>
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
4/17

2021/4/28 PostgreSQL File Manager
❖ Relations as Files (cont)
The relpath function maps RelFileNode to le:
char *relpath(RelFileNode r) // simplified
{
char *path = malloc(ENOUGH_SPACE);
if (r.spcNode == GLOBALTABLESPACE_OID) {
/* Shared system relations live in PGDATA/global */
Assert(r.dbNode == 0);
sprintf(path, “%s/global/%u”,
DataDir, r.relNode);
}
else if (r.spcNode == DEFAULTTABLESPACE_OID) {
/* The default tablespace is PGDATA/base */
sprintf(path, “%s/base/%u/%u”,
DataDir, r.dbNode, r.relNode);
}
else {
/* All other tablespaces accessed via symlinks */
sprintf(path, “%s/pg_tblspc/%u/%u/%u”, DataDir
r.spcNode, r.dbNode, r.relNode);
}
return path;
}
COMP9315 21T1 ♢ PG File Manager ♢ [4/15]
<< ∧ >>
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
5/17

2021/4/28 PostgreSQL File Manager
❖ File Descriptor Pool
Unix has limits on the number of concurrently open les. PostgreSQL maintains a pool of open le descriptors:
to hide this limitation from higher level functions to minimise expensive open() operations
Filenamesaresimplystrings:typedef char *FileName Openlesarereferencedvia:typedef int File
A File is an index into a table of “virtual le descriptors”.
COMP9315 21T1 ♢ PG File Manager ♢ [5/15]
<< ∧ >>
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
6/17

2021/4/28 PostgreSQL File Manager
❖ File Descriptor Pool (cont) Interface to le descriptor (pool):
File FileNameOpenFile(FileName fileName,
int fileFlags, int fileMode);
// open a file in the database directory ($PGDATA/base/…)
File OpenTemporaryFile(bool interXact);
// open temp file; flag: close at end of transaction?
void FileClose(File file);
void FileUnlink(File file);
int FileRead(File file, char *buffer, int amount);
int FileWrite(File file, char *buffer, int amount);
int FileSync(File file);
long FileSeek(File file, long offset, int whence);
int FileTruncate(File file, long offset);
Analogous to Unix syscalls open(), close(), read(), write(), lseek(), …
COMP9315 21T1 ♢ PG File Manager ♢ [6/15]
<< ∧ >>
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
7/17

2021/4/28 PostgreSQL File Manager
❖ File Descriptor Pool (cont) Virtual le descriptors (Vfd)
physically stored in dynamically-allocated array
also arranged into list by recency-of-use
<< ∧ >>
VfdCache[0] holds list head/tail pointers. COMP9315 21T1 ♢ PG File Manager ♢ [7/15]
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
8/17

2021/4/28 PostgreSQL File Manager
❖ File Descriptor Pool (cont) Virtual le descriptor records (simplied):
typedef struct vfd
{
s_short fd;
u_short fdstate;
File nextFree;
File lruMoreRecently; // doubly linked recency-of-use list
File lruLessRecently;
long seekPos; // current logical file position
char *fileName; // name of file, or NULL for unused VFD
// NB: fileName is malloc’d, and must be free’d when closing the VFD
int fileFlags; // open(2) flags for (re)opening the file
int fileMode; // mode to pass to open(2)
} Vfd;
COMP9315 21T1 ♢ PG File Manager ♢ [8/15]
<< ∧ >>
// current FD, or VFD_CLOSED if none
// bitflags for VFD’s state
// link to next free VFD, if in freelist
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
9/17

2021/4/28 PostgreSQL File Manager
❖ File Manager
Reminder: PostgreSQL le organisation
<< ∧ >>
COMP9315 21T1 ♢ PG File Manager ♢ [9/15]
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
10/17

2021/4/28 PostgreSQL File Manager
❖ File Manager (cont) PostgreSQL stores each table
in the directory PGDATA/pg_database.oid often in multiple les (aka forks)
<< ∧ >>
COMP9315 21T1 ♢ PG File Manager ♢ [10/15]
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
11/17

2021/4/28 PostgreSQL File Manager
❖ File Manager (cont) Datales (Oid,Oid.1,…):
sequence of xed-size blocks/pages (typically 8KB)
each page contains tuple data and admin data (see later) max size of data les 1GB (Unix limitation)
<< ∧ >>
COMP9315 21T1 ♢ PG File Manager ♢ [11/15]
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
12/17

2021/4/28 PostgreSQL File Manager
❖ File Manager (cont) Freespacemap (Oid_fsm):
indicates where free space is in data pages “free” space is only free after VACUUM
(DELETE simply marks tuples as no longer in use xmax) Visibilitymap (Oid_vm):
indicates pages where all tuples are “visible” (visible = accessible to all currently active transactions)
such pages can be ignored by VACUUM
COMP9315 21T1 ♢ PG File Manager ♢ [12/15]
<< ∧ >>
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
13/17

2021/4/28 PostgreSQL File Manager
<< ∧ >>
❖ File Manager (cont)
The “magnetic disk storage manager” (storage/smgr/md.c)
manages its own pool of open le descriptors (Vfd’s) may use several Vfd’s to access data, if several forks manages mapping from PageID to le+offset.
PostgreSQL PageID values are structured:
typedef struct
{
RelFileNode rnode; // which relation/file ForkNumber forkNum; // which fork (of reln) BlockNumber blockNum; // which page/block
} BufferTag;
COMP9315 21T1 ♢ PG File Manager ♢ [13/15]
cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html
14/17

2021/4/28 PostgreSQL File Manager
<< ∧ >>
❖ File Manager (cont)
Access to a block of data proceeds (roughly) as follows:
// pageID set from pg_catalog tables
// buffer obtained from Buffer pool getBlock(BufferTag pageID, Buffer buf) {
Vfd vf; off_t offset;
(vf, offset) = findBlock(pageID) lseek(vf.fd, offset, SEEK_SET) vf.seekPos = offset;
nread = read(vf.fd, buf, BLOCKSIZE)
if (nread < BLOCKSIZE) ... we have a problem } BLOCKSIZE is a global congurable constant (default: 8192) COMP9315 21T1 ♢ PG File Manager ♢ [14/15] cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html 15/17 2021/4/28 PostgreSQL File Manager ❖ File Manager (cont) findBlock(BufferTag pageID) returns (Vfd, off_t) { offset = pageID.blockNum * BLOCKSIZE fileName = relpath(pageID.rnode) if (pageID.forkNum > 0)
fileName = fileName+”.”+pageID.forkNum
if (fileName is not in Vfd pool)
fd = allocate new Vfd for fileName
else
fd = use Vfd from pool
if (pageID.forkNum > 0) {
offset = offset – (pageID.forkNum*MAXFILESIZE)
}
return (fd, offset)
}
COMP9315 21T1 ♢ PG File Manager ♢ [15/15]
<< ∧ cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html 16/17 2021/4/28 PostgreSQL File Manager Produced: 28 Feb 2021 cgi.cse.unsw.edu.au/~cs9315/21T1/lectures/pg-files/slides.html 17/17

Related Posts