ESS116
ESS 116
Introduction to Data Analysis in Earth Science
Image Credit: NASA
Instructor: Mathieu Morlighem
E-mail: mmorligh@uci.edu (include ESS116 in subject line)
Office Hours: 3218 Croul Hall, Friday 2:00 pm – 3:00 pm
This content is protected and may not be shared uploaded or distributed
Lecture 1 quick review
Lecture 2 – MATLAB’s File I/O vectors and matrices
Vectors and Matrices
Introduction to MATLAB’s visualization tools
File Input/Output
Today’s lecture
Lecture 1 – review
Absolute vs. Relative Paths
“..” is the parent directory (to move up “cd ..”)
“.” is the current directory
Any Path that starts with C: or / is ABSOLUTE
You are in “MATLAB” and want to go to “Shared”:
Absolute Path: >> cd /Users/Shared
Relative Path: >> cd ../../../Shared
Users
Guest
Shared
mmorligh
Adobe
Library
Desktop
Desktop
Downloads
scripts
Documents
Adobe
MATLAB
ISSM
seism
Pictures
C/C++
Microsoft
Apple
/ or C:
Library
Network
System
pwd: displays absolute path of the current directory
cd: to change directory
Either absolute or relative path
Use “../” to go back one directory
ls: lists files in current directory
Use wildcard “*” to select a subset of files:
ls *.txt
ls *data*.dat
whos: lists current variables in MATLAB’s workspace
rand: returns a random floating point number between 0-1
MATLAB functions to remember
Variable name is on the left, its value goes to the right
First character of variable name has to be a letter
>> A=2;
>> radius4=40.2e-5;
>> 2A=4; %DOES NOT WORK
Comments start with “%”
MATLAB functions have the following syntax:
>> output = FunctionName(argument1,argument2);
>> a2 = round(a);
>> [output1 output2] = FunctionName(argument1);
What is the difference between:
>> a=2
>> a=2;
MATLAB usage basics
i>Clicker question
You are in “Guest”. How do you copy and rename script.m in “Desktop” to script2.m in “Shared”?
copyfile(‘C:\Users\mmorligh\Desktop\script.m’,’C:\Users\Shared\script2.m’)
copyfile(‘C:\Users\mmorligh\Desktop\script.m’,’..\Shared\script2.m’)
copyfile(‘..\mmorligh\Desktop\script.m’ ,’C:\Users\Shared\script2.m’)
copyfile(‘..\mmorligh\Desktop\script.m’ ,’..\Shared\script2.m’)
All of the Above
Users
Guest
Shared
mmorligh
Adobe
Library
Desktop
Desktop
Downloads
scripts
Documents
Adobe
MATLAB
ISSM
seism
Pictures
C/C++
Microsoft
Apple
/ or C:
Library
Network
System
What does this MATLAB command do?
>> ls Documents\*.txt*
Lists all the files in the current directory whose name includes “.txt”
Lists all the files in the current directory that start with Documents and include “.txt” ?
Lists all the files in the directory “Documents” whose names include “.txt”
i>Clicker question
Lecture 2 – Input/Output, visualization,
vectors and matrices
Vectors and Matrices in MATLAB
MATLAB is short for MATrix LABoratory !
= 2×2 matrix (square)
= 4×2 matrix (rectangular)
Matrix construction follows specific mathematical rules:
Dimensions [m x n]
m = rows (vertical)
n = columns (horizontal)
All numerical quantities are technically matrices!
= 1×1 matrix (scalar)
= 2×1 matrix (column vector)
= 1×3 matrix (row vector)
Matrix Construction: The Basics
The matrix has you…
Defining arrays (vectors/matrices) in MATLAB is straightforward
Must use square brackets []
Separate numbers within a row → spaces or commas ,
Separate rows → semi-colon ;
rowVec =
>> rowVec=[1 2 3]
>> rowVec=[1,2,3]
Turn a row vector into a column vector using apostrophe ‘
Mathematically, this is called the transpose
>> columnVect = rowVect’
mat3x2 =
>> mat3x2 = [1 4; 2 5; 3 6]
>> mat3x2 = [1,4; 2,5; 3,6]
Defining Matrices
Matrix Addition/Subtraction
Both must have same dimensions
Add each corresponding element
Result has same dimensions
+ is commutative A + B = B + A
– is not commutative A – B ≠ B – A
Matrix Multiplication: 2 x 3 * 3 x 4 –> 2 x 4
Not commutative
MATLAB assumes all math operations are on matrices
For element by element operations use a dot before the symbol (arrays must have the same size!)
>> A * B %matrix multiplication
>> A .* B %array multiplication
>> A ./ B %array division
>> A .^ B %array to a power
Matrix Operations: The Basics
Often, we want to make long lists of numbers
>> [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20] %very tedious
>> [0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60]
>> [50 45 40 35 30 25 20 15 10 5 0 -5 -10 -15 -20 -25 -30] %boring!
We have two handy options in MATLAB to make lists
The colon operator
>> 0:3:60 %begin:increment:end
>> 1:20 %The increment is assumed to be 1 unless specified
>> 50:-5:-30 %note that the increment can be negative
“linspace” function: linspace(begin,end,n)
>> linspace(1,20,20) %Gives 20 evenly spaced values between 1-20
>> linspace(0,60,21) %Gives 21 evenly spaced values between 0-60
>> linspace(50,-30,17) %Gives 17 evenly spaced values between 50 & -30
Don’t these do the same thing? Why have both?
linspace and the Colon Operator
I want to create a vector that:
Starts with 1
Uses an increment of 3
Stops at 31?
>> linspace(1,3,31);
>> linspace(1,31,10);
>> 1:31
>> 1:3:31
i>Clicker question
test = test2=
To refer to a particular element of a matrix, use its row,col (subscripted)
>> test(3, 1) %This returns what value?
>> test2(4)
Can also refer to the linear index
For a matrix, can be quite confusing generally a bad idea
Useful for vectors!
>> test(3) %This returns what value?
>> test(10)
(row,col) notation: Linear index:
Referring to Matrix Elements
M =
What is the value of M(3,2) ?
5
1
10
6
i>Clicker question
M =
What is the value of M(4) ?
2
4
8
12
i>Clicker question
test = test2 =
To refer to an entire column of a matrix, use a colon in the col entry
>> test(:, 1) %This returns what value?
>> test(:, 3)
>> test2(:,1)
Can also refer to an entire row
>> test(1, 🙂 %This returns what value?
>> test(2, 🙂
>> test2(1,:)
This is a HUGE time saver. Use the colon!
Referring to Matrix Elements
Replacing/Removing Matrix Elements
test = test2 =
To change the 2 to 16 in test and test2:
>> test(1, 3) = 16
>> test2(3) = 16
Can also refer to an entire row
>> test(1, 🙂 = 5;
>> test(2, 🙂 = 10;
To remove an element, use an empty matrix “[]”
>> test2(3) = [] %what does this do?
>> test(2,3) = [] %why does this given an error?
>> test(:, 2) = [] %what does this do?
Finding Matrix Elements
test = test2 =
To find the locations of “2” in test:
>> pos = find(test==2); %linear indices
>> [i j] = find(test==2); %rows and columns
To find the locations where test>5
>> pos = find(test>5) ; %linear indices
>> [i j] = find(test>5); %rows and columns
We can combine find and replace:
>> pos = find(test>5);
>> test(pos) = 5;
Concatenating strings is easy!
>> str = ‘This ‘
>> str2 = ‘is ‘
>> str3 = ‘easy!’
>> finalStr = [str str2 str3]
>> newStr = [‘This’, ‘ ‘, ‘is ‘, ‘also’, ‘ easy! ‘]
Concatenating a matrix works the same way (make sure dimensions are consistent)
Row vector
>> stuff = [2 3 4]
>> otherStuff = [5 6 7]
>> allStuff = [stuff, otherStuff]
Column vector
>> stuff = [1; 2; 3]
>> otherStuff = [4; 5; 6; 7]
>> allStuff = [stuff; otherStuff]
Matrix
>> mat = [1 3; 2 4]
>> mat2 = [5 7; 6 8]
>> newMat = [mat mat2] %what will size of newMat be?
>> newMat2 = [mat; mat2] %what will size of newMat2 be?
Concatenating Matrices
Matrix Size and Dimensions
test = test2 = test3 =
Sometimes we want to know the length, dimensions, or number of elements in a matrix
To get the matrix dimensions (rows, cols) use “size”
Size returns a matrix, so you can store the rows/cols if you want
>> size(test)
>> [m, n] = size(test3)
>> [row, col] = size(test2)
To get vector length use “length”
>> len = length(test2)
Most functions that generate values, can generate matrices!
>> A=rand(3,4)
>> B=round(rand(6,2)*20)
What do you think these commands do? Try them out!
>> A=zeros(4,3)
>> B=ones(2,10)
>> C=(7*ones(3,4))’
Using Functions with Matrices
In Math class, you probably only learned about 2D matrices
MATLAB allows for multi-dimensional matrices
E.g. color images are typically stored in 3D matrices
How to define a 3D matrix? One slice at a time
>> mat3D(:, :, 1) = [9 4 2; 3 7 5; 11 1 10; 8 6 12]
>> mat3D(:, :, 2) = [2 1 9; 10 5 8; 12 4 3; 5 7 6]
>> mat3D(:, :, 3) = [3 0 10; 6 2 5; 9 4 8; 12 7 1]
What is the output below?
>> mat3D(2, 3, 2)
>> B = mat3D(2, :, 2)
>> new = mat3D(:, :, 3)
3D Matrices
mat3D =
Data Visualization
Plot one point:
% This is a really simple plot of just one point!
% Create coordinate variables and plot a red ‘*’
x = 11;
y = 48;
plot(x,y,’r*’)
% Change the axes and label them
axis([9 12 35 55])
xlabel(‘Time (hour)’)
ylabel(‘Temperature (C)’)
% Put a title on the plot
title(‘Temperature as a function of time’)
Creating plots
Plot more than one point:
%Make vectors for x and y
x = 1:0.2:10;
y = 2000*sin(x) + 5000;
%plot curve that connects these points
%as a red line
plot(x,y,’-r’)
Creating plots
Why is this plot unacceptable?
Plot more than one point:
%Make vectors for x and y
x = 1:0.2:10;
y = 2000*sin(x) + 5000;
%plot curve that connects these points
%as a red line
plot(x,y,’-r’)
%Add labels and title
xlabel(‘time (min)’);
ylabel(‘Earth”s core temperature’);
title(‘This is my first plot’);
Creating plots
Possible colors:
‘b’ (blue), ‘g’ (Green), ‘k’ (black), ‘r’ (red),…
Line types:
— (dashed), -. (dash dot), : (dotted), – (solid)
Markers:
o (circle), + (plus), * (asterisk), s (square), ^ (triangle),…
Customizing a plot
plot(x,y,’-.b’);
plot(x,y,’g*’);
plot(x,y,’*:r’);
Line and marker options:
‘LineWidth’ : Changes line width (default is 1)
‘MarkerFaceColor’: followed by a color (e.g. ‘r’)
‘MarkerEdgeColor’: followed by a color (e.g. ‘k’)
‘MarkerSize’: Changes marker size (default is 1)
Example:
plot(x,y,’r+:’,’MarkerFaceColor’,’g’,’MarkerSize’,5,’LineWidth’,2)
Some other options:
axis([xmin xmax ymin ymax])
grid on/grid off
legend(‘legend line 1′,’legend line 2’);
Customizing a plot
Plot more than one line: hold on
This plot is a bit overwhelming.
I just do this to demonstrate the capabilities of “plot”
%Make X-values
x = 1:0.2:10;
%Make Y-values
y = 2000*sin(x) + 5000;
%Make a second set of Y-values
y2 = (x-2).^2 .* (x-5).^2 .* (x-9).^2 * 10;
%Make a third set of Y-values
y3 = (x-1).^2 .* (x-4).^2 .* (x-7).^2 .* (x-9.5).^2;
%Clear current figure
clf;
%plot the data
plot(x,y,’b–‘,’LineWidth’,2);
%Allow the next plot to be on the same axes
hold on
%plot the second dataset
plot(x,y2,’ro-‘,’MarkerEdgeColor’,’k’,’MarkerFaceColor’,’r’);
%plot the third dataset
plot(x,y3,’^’,’MarkerEdgeColor’,’m’,’MarkerFaceColor’,’g’,’MarkerSize’,9);
%Label the plot
xlabel(‘time (min)’);
ylabel(‘Interest in MATLAB (%)’);
axis([1 10,-1000 10000]);
grid on
legend(‘Pr. Morlighem’,’Student 1′,’Student 2′);
Plot on multiple figures: figure
Multiple Plots In One Figure: subplot
Input-Output (I/O)
File I/O
Hard Disk
Memory (RAM)
File Input
File Output
Where variables are stored
Temporary: Goes away when computer is shut off
Fast to read/manipulate
Where files are stored
Permanent: Stays even after computer is shut off
Slow to read/manipulate
File Input: load: loads in an ASCII file (or .mat) file
File must have consistent number of columns
All data must be numeric; no formatting options (loaded as class “double”)
File Input
Advanced National Seismic System (ANSS)
http://www.ncedc.org/anss/
Near real-time data for all global earthquake events!
Not just numbers
Common Earth Science Data Sets
USGS Current Water Data
http://waterdata.usgs.gov/nwis/rt
Near real-time data for streams in all 50 states!
Not just numbers
load can’t read files with numbers and strings
What if you want to read a file that has both numbers and words/characters in a single row?
textscan can read files with mixed content
Data is stored in a cell array (containing individual columns)
Each cell can contain any type of variable (char, double, int16)
File must already be opened with fopen
Must close file with fclose when finished
File Input: textscan
Always open the file first to see what it looks like
What are the columns made of:
integers (%d),
floating point numbers (%f)
text (%s)
Example: rocks.dat
File Input: textscan
81.472 granite a
90.579 rhyolite b
12.699 diorite c
91.338 andesite d
63.236 basalt e
9.754 gabbro f
%d %f %s %s
%open file
file = ‘rocks.dat’;
fid = fopen(file);
%read file
cellMat = textscan(fid,’%d %f %s %s’);
%close file
fclose(fid);
File Input: textscan
fopen: opens a file and returns a “File Identifier” used to identify the file later on in your code (-1 signifies an error in opening the file)
fclose: After you are finished with a file, you should close it (prevents errors and unwanted behavior).
textscan: read all the lines of the file and populate variable ‘cellMat’ (one element per column)
%d: integer (e.g., 235)
%f: floating-point number (e.g., 20.345)
%s: string (e.g., word)
Cell Arrays: Content Indexing
How do I create a cell array?
Cell arrays are created using curly braces { }
Same indexing rules as matrices
To get a cell’s contents use curly braces { }
Returns the cell’s contents of the specified index
%open file
file = ‘rocks.dat’;
fid = fopen(file);
%read file
cellMat = textscan(fid,’%d %f %s %s’);
%close file
fclose(fid);
%extract data
number = cellMat{1};
density = cellMat{2};
name = cellMat{3};
letter = cellMat{4};
%From now on, let’s forget about cellMat 🙂
File Input: textscan
Loading data from a file
load(‘filename’)
not covered in this course
File has a .mat extension
fopen,textscan,fclose
no (and yes)
Only contains numbers
Consistent number of columns
no
yes
Is it an ASCII (text) file?
yes
yes
no
data = load(‘filename’)
MATLAB Commands to remember
Lab 2: Vectors and Matrices
DUE: one week after the lab starts (canvas)
Bring a USB drive
Lecture 3: Programming
What’s next ?
1
2
�
⇥
1
⇤
⇥
1 2 3
⇤
1 3
2 4
�
2
66
4
1 5
2 6
3 7
4 8
3
77
5
2
4
1 4
2 5
3 6
3
5
⇥
1 2 3
⇤
2
66
4
9 4 2
3 7 5
11 1 10
8 6 12
3
77
5
2
66
4
(1, 1) (1, 2) (1, 3)
(2, 1) (2, 2) (2, 3)
(3, 1) (3, 2) (3, 3)
(4, 1) (4, 2) (4, 3)
3
77
5
2
66
4
(1) (5) (9)
(2) (6) (10)
(3) (7) (11)
(4) (8) (12)
3
77
5
⇥
3 8 2 �4 6 1
⇤
2
4
10 41
24 2
5 12
3
5
2
66
4
9 4 2
3 7 5
11 1 10
8 6 12
3
77
5
2
66
4
2 1 9
10 5 8
12 4 3
5 7 6
3
77
5
2
66
4
3 0 10
6 2 5
9 4 8
12 7 1
3
77
5
ESS116: MATLAB Cheat Sheet
1 Path and file operations
cd Change Directory (followed by absolute or relative path of a directory)
cd ../../Shared (relative path)
cd /Users/Shared (absolute path)
pwd display current directory’s absolute path (Path Working Directory)
ls display list of files and directories in the current directory
(can be followed by a path and/or file name pattern with *)
ls ../file*mat
ls *.txt
ls /Users/mmorligh/Desktop/
copyfile copy existing file into a new directory, and/or rename a file
copyfile(‘/Users/Shared/foo.txt’,’.’);
copyfile(‘foo.txt’,’bar.txt’);
mkdir create a directory
mkdir Lab1
2 Fundamental MATLAB classes
double floating point number (1.52, pi, …) → MATLAB’s default type
int8 Integer between -128 and 127 (8 bits, saves memory)
uint8 Unsigned integer between 0 and 255 (used primarily for images)
int16 Integer between -32768 and 32767 (16 bits)
logical true/false
string data type for text (str = ‘This is a string’;)
cell cell array, used by textscan
3 Matrices
Use square [] to create a matrix, and ; to separate rows
A=[1 2 3;4 5 6;7 8 9];
ones, zeros create a matrix full of ones or zeros
A=ones(5,2);
‘ transpose a matrix
B=A’;
length return length of a vector (do not use for matrices)
size returns the size of a matrix
(number of rows then columns, then 3rd dimension if 3D, etc)
[nrows,ncols]=size(A); [nrows,ncols,nlayers]=size(A3D);
linspace and : to create vectors
A=2:3:100;
A=linspace(2,100,10);
find return the linear indices where a condition on the elements of a matrix is met
pos=find(A==−9999);
pos=find(A>100);
Extract the first 10 even columns of a matrix
B=A(:,2:2:20);
Removing elements: use empty brackets
A(:,2)= [];
Concatenate matrices
A=’This is ‘; B=[A ‘an example’];
Replacing elements in a matrix (use either linear or row,col notation)
A(10,3)=5.5;
pos=find(A==−9999);
A(pos)= 0;
Element-by-element operation: use a dot (.) before the operator
A= C.*D;
4 I/O
load loads a MATLAB file (*.mat) into the workspace, or a text file with only numbers
and consistent number of columns
load(‘data.mat’);
data=load(‘data.txt’);
textscan loads a text file into a cell array (as many elements as there are columns in the file)
Use %d for integers, %f for floating point numbers %s for strings
fid = fopen(‘filename’);
data = textscan(fid,’%d %f %s %s’,’Headerlines’,5);
fclose(fid);
%Put first column in A, and second column in B
A = data{1}; B = data{2};
5 fprintf
fprintf print text (and variables) to the screen. First argument is a string with placeholders.
fprintf(‘The radius is %7.2f and A = %d !!\n’,EarthRadius,10);
– Special characters: \n (new line) %% (percent sign) ” (apostrophe)
– Variable specifiers: %s (string) %d (integer) %e (exponential) %f (float)
– %010.3f: leading 0, 10 total spaces, 3 decimals. Ex: 000003.142
6 Visualization
plot displays a list of points (x,y)
plot(x,y,’−r’);
plot(x,y,’r+:’,’MarkerFaceColor’,’g’,’MarkerSize’,5,’LineWidth’,2);
axis controls x and y axes
axis([xmin xmax ymin ymax]);
axis equal tight
legend adds a legend to previously plotted curves
legend(‘First curve’,’second curve’)
figure creates a new figure window
figure(2)
xlabel/ylabel/title control x/y axis labels and plot title
xlabel(‘Distance (km)’);
hold on keep current plot so that whatever follows is plotted on the same plot
subplot divide figure into several subplots
subplot(2,3,1)
histogram make a histogram for a vector
histogram(tmax,20);
histogram(tmax,round(sqrt(length(tmax))));
7 Relational and Logical operators
== equal to, ~= not equal to, > greater than, >= greater than or equal to,
< less than, <= less than or equal to.
&& and, || or, ~ not.
A=( (1>10)|| (3~=4));
8 If/elseif/else and for loops (examples)
Counting algorithm
%Initialize counters
counter1 = 0;
counter2 = 0;
%Go over all of the elements of T and increment counters when a condition is met
for i=1:length(T)
if T(i)>100
counter1 = counter1+1
elseif T(i)<0
counter2 = counter2+1
end
end
fprintf('Found %d days with T>%f, and %d with T<%f\n',counter1,100,counter2,0)
Extracting (after counting!)
%You first need to count how many times T>100 (for example), then: allocate memory
hotdays = zeros(counter1,1);
%Go through T, again, and store temperatures>100 in hotdays
count = 1;
for i=1:length(T)
if T(i)>100
hotdays(count)=T(i);
count = count+1;
end
end
9 Statistic
mean computes mean of a vector
median computes median of a vector
std computes standard deviation
min returns minimum value in a vector
max returns minimum value in a vector
skewness returns skewness
kurtosis returns kurtosis
normcdf/tcdf/chi2cdf cumulative density function for a normal, t and χ2 distributions
norminv/tinv/chi2inv inverse of the cumulative density function
p0 = normcdf(x0,mu,sigma);
x0 = norminv(p0,mu,sigma);
p0 = tcdf(x0,V);
x0 = tinv(p0,V);
10 Polynomials and interpolations
polyval returns the value of a polynomials (represented by its coefficient) for some x
coeff = [3 2.7 1 −5.7];
x=0:0.2:0.6;
y=polyval(coeff,x)
polyfit returns the coefficient of the polynomials that best fit data points
coeff = polyfit(datax,datay,3); %3 means cubic polynomial
interp1 interpolates between data points (spline or linear)
y1=interp1(datax,datay,’linear’);
y2=interp1(datax,datay,’spline’);
11 Image processing
imread loads an image (as a matrix) into the workspace
A=imread(‘image.png’)
image display an image (2D or 3D)
imagesc display a 2D image and scale indices to use all the colors in the color map
colormap set a colormap (only for indexed images)
Indexed Images (2D)
Indexed image, need two matrices:
iMat a 2D matrix with indices
cMap is a nx3 2D matrices with the RGB code for each index
To display this image:
image(iMat);
colormap(cMap);
If the colormap is not consistent with indices, you need to use imagesc(iMat).
True-color image (3D, RGB)
No need to prescribe a colormap:
iMat(:,:,1) Red matrix (between 0–255 if uint8, or 0–1 if double)
iMat(:,:,2) Green matrix
iMat(:,:,3) Blue matrix
To display this image: image(iMat).
12 Miscellaneous
rand returns a random floating point number between 0 and 1
x=rand
x=rand(10,2)
round round input to closest integer
A=round(rand*10);
whos displays list of all variables in MATLAB workspace
tic/toc displays cpu time for a chunk of code
sqrt square root
13 Functions
Calling a function: [output1,output2] = functionname(arg1,arg2);
Function header (top lines of the file that implements this function):
function [output1,output2] = functionname(arg1,arg2)
% H1 line: describe what the function does
ESS116, M. Morlighem, Updated: March 14, 2019