1.Computer Vision Pipeline（计算机视觉管道）
预处理主要是关于标准化数据，比如处理输入图像大小。
Separating Data（分离数据）
Images as Grids of Pixels
Import resources
import numpy as np
import matplotlib.image as mpimg # for reading in images
import matplotlib.pyplot as plt
import cv2 # computer vision library
%matplotlib inline
Read in and display the image
# Read in the image
image = mpimg.imread('images/waymo_car.jpg')
# Print out the image dimensions
print('Image dimensions:', image.shape)
# Change from color to grayscale
gray_image = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY)
plt.imshow(gray_image, cmap='gray')
# Create a 5x5 image using just grayscale, numerical values
tiny_image = np.array([[0, 20, 30, 150, 120],
[200, 200, 250, 70, 3],
[50, 180, 85, 40, 90],
[240, 100, 50, 255, 10],
[30, 0, 75, 190, 220]])
# To show the pixel grid, use matshow
plt.matshow(tiny_image, cmap='gray')
RGB colorspace
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
%matplotlib inline
# Read in the image
image = mpimg.imread('images/wa_state_highway.jpg') a
plt.imshow(image)
RGB channels
Visualize the levels of each color channel. Pay close attention to the traffic signs!
# Isolate RGB channels
r = image[:,:,0]
g = image[:,:,1]
b = image[:,:,2]
# Visualize the individual color channels
f, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(20,10))
ax1.set_title('R channel')
ax1.imshow(r, cmap='gray')
ax2.set_title('G channel')
ax2.imshow(g, cmap='gray')
ax3.set_title('B channel')
ax3.imshow(b, cmap='gray')
编码蓝屏应用
色彩空间
我们已经知道要怎么检测蓝幕背景了,但这种检测方法是有前提的,那就是场景光线要好 而且蓝幕的颜色得十分连贯,如果光线发生了变化墙壁有阴影、很斑驳或太亮了怎么办？这时简单的蓝色阀值就不适用了。
那我们要如何完整地检测出处于不同光线下的物体呢？
其实 表示图像颜色的方法还有很多，不仅仅有RGB这种颜色分量。
我们通常把各种各样的颜色表示法称为“颜色空间”
RGB
R\G\B三维坐标来表示，比如白色坐标为（255，255，255）
HSV
三个字母分别表示色相、饱和度、明度
HLS
则是指色相、亮度、饱和度
以上就是图像处理最常用的几种颜色空间
利用HSV颜色空间进行图像处理
分离出每个像素的明度，即Value（明度），明度受照明条件的影响最大。
H通道基本不受阴影或过高亮度影响，如果们用H通道，舍弃V通道信息，那就能对彩色物体进行检测，而且效果会比在RGB颜色空间更为可靠
依靠HSV检测粉色气球
标准化输出
分类数值转换为数值：
- 整数编码
整数编码意味着每个类别分配一个整数值，比如：day = 0;night = 1;
- one hot-encoding 独热编码
独热编码通常用于超过两类。例如：由于我们有四个类（猫、虎、河马、 狗），独热编码为[0, 0, 0, 1]一个列表
数据标准化
提取特征
使用HSV色彩空间提取平均亮度作为特征，具体来说 我们会使用明度 (value) 通道它用来测量亮度，接下来 把总和除以图像面积，得到图像的平均亮度
#RGB to HSV
image_num = 0
test_im = STANDARDIZED_LIST[image_num][0]
test_label = STANDARDIZED_LIST[image_num][1]
# Convert to HSV
hsv = cv2.cvtColor(test_im, cv2.COLOR_RGB2HSV)
# Print image label
print('Label: ' + str(test_label))
# HSV channels
h = hsv[:,:,0]
s = hsv[:,:,1]
v = hsv[:,:,2]
# Plot the original image and the three channels
f, (ax1, ax2, ax3, ax4) = plt.subplots(1, 4, figsize=(20,10))
ax1.set_title('Standardized image')
ax1.imshow(test_im)
ax2.set_title('H channel')
ax2.imshow(h, cmap='gray')
ax3.set_title('S channel')
ax3.imshow(s, cmap='gray')
ax4.set_title('V channel')
ax4.imshow(v, cmap='gray')
在本例中单独绘制 H、S 和 V 通道，这是一张白天的图像 以及不同颜色通道 H、S、V，我们可以看到 V 通道的天空亮度特别高，利用 V 通道确定平均亮度。
定义一个函数来找到图像的平均值，函数avg_brightness 会读入一个 RGB 图像：
1.把图像转换为 HSV 颜色空间
2.对 V 通道的所有像素值求和
3.计算图像面积，这里是 600 乘以 1100,将亮度总和除以图像的面积
# Find the average Value or brightness of an image
def avg_brightness(rgb_image):
# Convert image to HSV
hsv = cv2.cvtColor(rgb_image, cv2.COLOR_RGB2HSV)
# Add up all the pixel values in the V channel
sum_brightness = np.sum(hsv[:,:,2])
## TODO: Calculate the average brightness using the area of the image
# and the sum calculated above
area = 600 * 1100
avg = sum_brightness / area
return avg
# Testing average brightness levels
# Look at a number of different day and night images and think about
# what average brightness value separates the two types of images
image_num = 190
test_im = STANDARDIZED_LIST[image_num][0]
avg = avg_brightness(test_im)
print('Avg brightness: ' + str(avg))
plt.imshow(test_im)
分类器
#Import resources
import cv2 # computer vision library
import helpers
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
%matplotlib inline
# Training and Testing Data
# Image data directories
image_dir_training = "day_night_images/training/"
image_dir_test = "day_night_images/test/"
# Load the datasets
# Using the load_dataset function in helpers.py
# Load training data
IMAGE_LIST = helpers.load_dataset(image_dir_training)
# Standardize all training images
STANDARDIZED_LIST = helpers.standardize(IMAGE_LIST)
# Visualize the standardized data
# Display a standardized image and its label
# Select an image by index
image_num = 0
selected_image = STANDARDIZED_LIST[image_num][0]
selected_label = STANDARDIZED_LIST[image_num][1]
# Display image and data about it
plt.imshow(selected_image)
print("Shape: "+str(selected_image.shape))
print("Label [1 = day, 0 = night]: " + str(selected_label))
## Feature Extraction
def avg_brightness(rgb_image):
# Convert image to HSV
hsv = cv2.cvtColor(rgb_image, cv2.COLOR_RGB2HSV)
# Add up all the pixel values in the V channel
sum_brightness = np.sum(hsv[:,:,2])
area = 600*1100.0 # pixels
# find the avg
avg = sum_brightness/area
return avg
# Testing average brightness levels
# Look at a number of different day and night images and think about
# what average brightness value separates the two types of images
# As an example, a "night" image is loaded in and its avg brightness is displayed
image_num = 190
test_im = STANDARDIZED_LIST[image_num][0]
avg = avg_brightness(test_im)
print('Avg brightness: ' + str(avg))
plt.imshow(test_im)
# This function should take in RGB image input
def estimate_label(rgb_image):
## TODO: extract average brightness feature from an RGB image
# Use the avg brightness feature to predict a label (0, 1)
predicted_label = 0
avg = avg_brightness(rgb_image)
## TODO: set the value of a threshold that will separate day and night images
if avg > 110:
predicted_label = 1
else:
predicted_label = 0
## TODO: Return the predicted_label (0 or 1) based on whether the avg is
# above or below the threshold
return predicted_label
# Test dataset
import random
# Using the load_dataset function in helpers.py
# Load test data
TEST_IMAGE_LIST = helpers.load_dataset(image_dir_test)
# Standardize the test data
STANDARDIZED_TEST_LIST = helpers.standardize(TEST_IMAGE_LIST)
# Shuffle the standardized test data
random.shuffle(STANDARDIZED_TEST_LIST)
def get_misclassified_images(test_images):
# Track misclassified images by placing them into a list
misclassified_images_labels = []
# Iterate through all the test images
# Classify each image and compare to the true label
for image in test_images:
# Get true data
im = image[0]
true_label = image[1]
# Get predicted label from your classifier
predicted_label = estimate_label(im)
# Compare true and predicted labels
if(predicted_label != true_label):
# If these labels are not equal, the image has been misclassified
misclassified_images_labels.append((im, predicted_label, true_label))
# Return the list of misclassified [image, predicted_label, true_label] values
return misclassified_images_labels
