Dive into DeepLearning - 02 - Preliminaries

Course Note: d2l-video-05 - 线性代数
Jupyter Notebook: chapter_preliminaries/linear-algebra.ipynb

预备知识中 Liner Algebra 的部分

线性代数

Scalars 标量: 指只有一个元素的张量 tensors
PYTHON
```
import torch
x = torch.tensor(3.0) # scalar
y = torch.tensor(2.0)
```
Click to expand and view more
Vectors 向量: 可以视作标量构成的列表
PYTHON
```
x = torch.arange(4)
x[3] # 通过张量索引访问任一元素
len(x) # 访问张量长度
x.shape # torch.Size([4]) 只有一个轴的张量, 形状只有一个元素
```
Click to expand and view more
Matrices 矩阵: 类似向量的推广, 可以构建更多轴的数据结构
PYTHON
```
# 构建矩阵
A = torch.arange(20).reshape(5, 4)
A.T # 转置

# 对称矩阵
B = torch.tensor([[1, 2, 3], [2, 0, 4], [3, 4, 5]])
B = B.T
```
Click to expand and view more
形状相同张量的计算
PYTHON
```
A = torch.arange(20, dtype=torch.float32).reshape(5, 4)
B = A.clone()
A, A + B
A * B # 对应元素相乘: Hadamard 积
```
Click to expand and view more
计算元素的和
PYTHON
```
x = torch.arange(4, detype=torch.float64)
x.sum() # 任意形状张量的和
```
Click to expand and view more
计算平均值
PYTHON
```
A.mean() # 均值
A.sum() / A.numel() # 另一种计算均值的方法: 和 / 数量
```
Click to expand and view more
点乘是相同位置元素乘积的和
PYTHON
```
x = torch.tensor([0., 1., 2., 3.]])
y = torch.tensor([1., 1., 1., 1.]])
torch.dot(x, y) # torch(6.)

# 或者通过元素乘法, 求和表示点积
torch.sum(x * y) # torch(6.)
```
Click to expand and view more
降维: axis 指定沿着哪一个轴来降低纬度
假如现在有个张量A如下
PLAINTEXT
```
tensor([[ 0.,  1.,  2.,  3.],
        [ 4.,  5.,  6.,  7.],
        [ 8.,  9., 10., 11.],
        [12., 13., 14., 15.],
        [16., 17., 18., 19.]])
```
Click to expand and view more
现在沿着第0轴, 通过求和降低纬度
PYTHON
```
A_sum_axis0 = A.sum(axis=0)
A_sum_axis0, A_sum_axis0.shape
```
Click to expand and view more
输出如下
上面降维的原理就是, 由于axis=0, 就将最外层的纬度去掉, 原来 A.shape=torch.Size([5, 4]), 现在变成了A_sum_axis0=torch.Size([4])
PLAINTEXT
```
(tensor([40., 45., 50., 55.]), torch.Size([4]))
```
Click to expand and view more
类似的, 还可以降低多个纬度
PYTHON
```
A.sum(axis=[0, 1])
```
Click to expand and view more
由于A就两个轴, 两个轴都被降低就成了标量
PLAINTEXT
```
tensor(190.)
```
Click to expand and view more
此外, 还可以保持纬度不变, 将要降的纬度变成1
PYTHON
```
sum_A = A.sum(axis=1, keepdims=True) # keepdims=True 不丢掉原来的纬度
```
Click to expand and view more
输出如下:
原来 A.shape=torch.Size([5, 4]) 现在变成了sum_A.shape=torch.Size([5,1])
PLAINTEXT
```
tensor([[ 6.],
        [22.],
        [38.],
        [54.],
        [70.]])
```
Click to expand and view more
这种机制常用于广播, 广播要求纬度相同, 例如 A / sum_A 的计算结果如下
PLAINTEXT
```
tensor([[0.0000, 0.1667, 0.3333, 0.5000],
        [0.1818, 0.2273, 0.2727, 0.3182],
        [0.2105, 0.2368, 0.2632, 0.2895],
        [0.2222, 0.2407, 0.2593, 0.2778],
        [0.2286, 0.2429, 0.2571, 0.2714]])
```
Click to expand and view more
还可以通过某个轴计算A元素的累积总和 A.cumsum(axis=0)
PLAINTEXT
```
tensor([[ 0.,  1.,  2.,  3.],
        [ 4.,  6.,  8., 10.],
        [12., 15., 18., 21.],
        [24., 28., 32., 36.],
        [40., 45., 50., 55.]])
```
Click to expand and view more
Norms 范数
范数可以理解为"向量的长度/大小"的一种度量方式,
- 向量范数
L1范数, 它表示为向量元素的绝对值之和 (曼哈顿距离)
PYTHON
```
torch.abs(u).sum()
```
Click to expand and view more
L2范数是向量元素平方和的平方根 (欧几里德距离)
PYTHON
```
u = torch.tensor([3.0, -4.0])
torch.norm(u)
```
Click to expand and view more
- 矩阵范数: 最小的满足下面公式的值
$$ c = A \cdot b \quad \text{hence} \quad |c| \leq |A| \cdot |b| $$
矩阵的Frobenius范数(Frobenius norm)是矩阵元素平方和的平方根
$$ |A|_{Frob} = \left(\sum_{i,j} A_{ij}^2 \right)^{\frac{1}{2}} $$
PYTHON
```
torch.norm(torch.ones((4, 9)))
```
Click to expand and view more

线性代数

Related Posts

Dive into DeepLearning - 01 - Preliminaries

Start searching

No results found