程序代写代做代考 python matlab algorithm CMP3103M AMR 3 – Sensing and Vision.key

CMP3103M AMR 3 – Sensing and Vision.key

!



CMP3103M AMR

Dr. Marc Hanheide

!

SYLLABUS
‣ Introduction to Robotics

‣ Robot Programming

‣ Robot Sensing & Vision

‣ Robot Control

‣ Robot Behaviours

‣ Control Architectures

‣ Navigation Strategies

‣ Map Building

Let’s make a robot following a
line!

?RECAP:
WHICH (TYPE OF) SENSORS DO EXIST AND ARE BEING USED?

!

QUICKVOTE

http://lncn.eu/cmp3103m

!

RECAP SENSORS

‣ Which of the following are “Exteroceptive” sensors?

A. Laser scanner

B. Wheel Encoder

C. CPU temperature sensor

D. Kinect

E. Gyrometer

F. Sonar

!

!

RECAP SENSORS

‣ Which approach does the Kinect 1 implement to see depth?

A. Depth from (de-)focus

B. Stereo vision

C. Time of Flight

D. Structured light

E. Ultrasound

F. Infrared

?PROPRIOCEPTION

!

FORCE / TORQUE

‣ Strain Gauge:

‣ Deformation changes
resistance

‣ put it on something “bendy”

‣ Calibration is an issue> 

R => Strain ?

POSITION

!

ODOMETRY IN TURTLEBOT
> rostopic list
> rostopic info /turtlebot_1/odom
> rosmsg show nav_msgs/Odometry 

…
geometry_msgs/PoseWithCovariance pose
geometry_msgs/Pose pose
geometry_msgs/Point position
float64 x
float64 y
float64 z
geometry_msgs/Quaternion orientation
float64 x
float64 y
float64 z
float64 w
float64[36] covariance
geometry_msgs/TwistWithCovariance twist
geometry_msgs/Twist twist
geometry_msgs/Vector3 linear
float64 x
float64 y
float64 z
geometry_msgs/Vector3 angular
float64 x
float64 y
float64 z
float64[36] covariance

https://github.com/LCAS/teaching/blob/indigo-devel/cmp3641m-code-fragments/scripts/odom_reader.py

https://github.com/LCAS/teaching/blob/indigo-devel/cmp3641m-code-fragments/scripts/odom_reader.py

?EXTEROCEPTION

DISTANCE

!

!

!

LASER MEASUREMENTS

‣ mapping

‣ obstacle avoidance

‣ tracking

‣ expensive! So “emulated” in
Turtlebot

TURTLEBOT “LASER”

SOME “LASER” CODE

!

RECAP ROS

‣ Which middleware paradigm(s) do “ROS topics” implement?

A. data push

B. data pull

C. publish-subscribe

D. event-driven processing

E. client-server

F. synchronous data processing

G. asynchronous data processing

!

RECAP ROS

‣ What is a ROS node?

A. a running process, using the ROS middleware

B. a piece of source code that uses the ROS library

C. a running process on a computer with ROS installed

D. the component also called “roscore”

E. a running python process, containing this line of code:
‣ rospy.init_node(‘whatever’)

!

WHAT IS WRONG WITH
THIS CODE?

‣ What is wrong?

A. wrong type for
subscriber topic

B. missing import
statement

C. publisher has no
topic to publish to

D. for construct has
syntax error

E. the node name is
invalid

import rospy
from std_msgs.msg import String
from sensor_msgs.msg import LaserScan

class FirstSub:
def __init__(self):
self.sub = rospy.Subscriber(‘/turtlebot_2/scan’,
Odometry,
callback=self.callback)
self.pub = rospy.Publisher(String)

def callback(self, msg):
for range in msg.ranges:
if range < 1.0: print "ALERT" s = String() s.data = "ALERT" self.pub.publish(s) rospy.init_node("firstsubscriber") fp = FirstSub() rospy.spin() import rospy from std_msgs.msg import String from sensor_msgs.msg import LaserScan class FirstSub: def __init__(self): self.sub = rospy.Subscriber('/turtlebot_2/scan', LaserScan, callback=self.callback) self.pub = rospy.Publisher('/warning', String) def callback(self, msg): for range in msg.ranges: if range < 1.0: print "ALERT" s = String() s.data = "ALERT" self.pub.publish(s) rospy.init_node("firstsubscriber") fp = FirstSub() rospy.spin() ‣ What is wrong? A. wrong type for subscriber topic B. missing import statement C. publisher has no topic to publish to D. for construct has syntax error E. the node name is invalid VISION AIM FOR TODAY Morgan Quigley, Brian Gerkey & And William D. Smart. (2015) Programming Robots with ROS [Chapter 12] ! ROBOT VISION ‣ Steps ‣ acquisition ‣ pre-processing ‣ segmentation ‣ feature extraction ‣ pattern recognition ‣ robot control 
 ‣ Requirements for algorithms ‣ Fast (real-time) ‣ Robust (to noise and changes in environment) ‣ Non-stationary assumption (limited use of background subtraction methods) ! ROBOT VISION ‣ Steps ‣ acquisition ‣ pre-processing ‣ segmentation ‣ feature extraction ‣ pattern recognition ‣ robot control 
 ‣ Requirements for algorithms ‣ Fast (real-time) ‣ Robust (to noise and changes in environment) ‣ Non-stationary assumption (limited use of background subtraction methods) ! ! OPENCV ‣ Initially launched by Intel in 1999 ‣ developed into the “gold standard” in computer vision processing ‣ cross-platform, multi-language ‣ Applications: ‣ 2D and 3D feature toolkits ‣ Egomotion estimation ‣ Facial recognition system ‣ Gesture recognition ‣ Human–computer interaction (HCI) ‣ Mobile robotics ‣ Motion understanding ‣ Object identification ‣ Segmentation and recognition ‣ Stereopsis stereo vision: depth perception from 2 cameras ‣ Structure from motion (SFM) ‣ Motion tracking ‣ Augmented reality ! B G REPRESENTATION OF IMAGES IN OPENCV ‣ Matrices like in Matlab! ‣ based on numpy ‣ not RGB, but BGR!? R G BR ! BASIC OPERATIONS IN OPENCV ‣ Pixel access: ‣ img[X, Y, P] ‣ img[3, 0, 2] ‣ img[0,1,0] ‣ Ranges: ‣ img[1:3, 2:3, 0] (3,0) R: P=2 G: P=1 (0,0) (1,0) (2,0) (3,0) (0,1) (3,3) B: P=0 ! IMAGES IN PYTHON / OPENCV iterate over pixels manipulation pixels read and show images ! DISPLAYING AND DRAWING IN OPENCV ! GETTING IMAGE STREAMS FROM A ROS TOPIC ‣ OpenCV and ROS play nicely ‣ There is a dedicated CvBridge to help you getting and processing image from the robot http://wiki.ros.org/cv_bridge/Tutorials/ConvertingBetweenROSImagesAndOpenCVImagesPython subscribe on Image topic convert to OpenCV in callback process the image using OpenCV http://wiki.ros.org/cv_bridge/Tutorials/ConvertingBetweenROSImagesAndOpenCVImagesPython ! A FIRST EXERCISE: 
 COLOUR SLICING http://www.pyimagesearch.com/2014/08/04/opencv-python-color-detection http://www.pyimagesearch.com/2014/08/04/opencv-python-color-detection ! ! ! ! ! ! ! COLOUR SLICING IN PYTHON subscribe on Image topic convert to OpenCV in callback optional: convert to different colour space (from BGR) use inRange for colour slicing display binary image optional: post-process image (morphological ops) ! COLOUR SLICING IN PYTHON ‣ What colour space to slice in? ‣ What values to choose? ‣ grab frames:
 rosrun image_view image_saver image:=/camera/rgb/image_color ‣ use “gimp” and its color picker ‣ use http://imagecolorpicker.com/ and upload an image http://imagecolorpicker.com/ ! A SECOND EXERCISE: FINDING CONTOURS ‣ From colour slicing to regions! ‣ Work through http://opencv-python-tutroals.readthedocs.org/ en/latest/py_tutorials/py_imgproc/py_contours/ py_contours_begin/py_contours_begin.html#contours-getting- started http://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_imgproc/py_contours/py_contours_begin/py_contours_begin.html#contours-getting-started ! A THIRD EXTRA EXERCISE: FINDING CIRCLES ‣ Homework: Hough Transform ‣ What is it? NOW FOR THE REAL STUFF Morgan Quigley, Brian Gerkey & And William D. Smart. (2015) Programming Robots with ROS [Chapter 12] ! RESOURCES ‣ https://github.com/osrf/rosbook (https://github.com/osrf/ rosbook/tree/master/followbot) ‣ catkin_init_workspace ‣ git clone https://github.com/osrf/rosbook.git ‣ catkin_make ‣ Morgan Quigley, Brian Gerkey & And William D. Smart. (2015) Programming Robots with ROS [Chapter 12] https://github.com/osrf/rosbook https://github.com/osrf/rosbook/tree/master/followbot https://github.com/osrf/rosbook.git SOME MORE ROBOT- RELATED VISION TO DISCUSS ! APPLICATIONS ‣ Visual Path Following, Lagadic project, INRIA, France http://www.youtube.com/watch?v=_t5cYUfLq1E ! FEATURES ‣ Features ‣ compact, meaningful representation of the image ‣ What are the good features? ‣ edges, corners, blobs, colour, texture ‣ State of the art ‣ SIFT, SURF, Haar-like, HOG, etc. 
 All in openCV! ‣ Applications - examples ‣ Object detection: template matching ‣ Scene reconstruction: extract depth information ! VISION IN ROBOTICS ! ! VISION IN ROBOTICS ! THANK YOU FOR LISTENING!