Week 11: Object Following with YOLO26

📋

Laboratory Overview

↑ Go Up

This laboratory introduces you to advanced autonomous robotics by implementing real-time object detection and following capabilities on the JetBot platform using YOLO26 — the latest generation of the You Only Look Once detection family. Building upon the fundamental collision avoidance techniques learned in Week 10, you will now enable your robot to intelligently identify, track, and follow specific objects in its environment. Using the state-of-the-art YOLO26 neural network pre-trained on the COCO (Common Objects in Context) dataset, your JetBot will be capable of detecting and classifying 80 different common objects in real-time, from people and animals to vehicles and household items.

The laboratory provides hands-on experience with computer vision for autonomous navigation, combining object detection, coordinate estimation, and motor control to create sophisticated tracking behavior. You'll learn how modern autonomous systems perceive their environment, make decisions based on visual input, and execute appropriate actions—skills essential for applications ranging from warehouse robots to autonomous vehicles.

What You'll Learn

YOLO26 Object Detection: Implement state-of-the-art real-time object detection using the latest pre-trained YOLO26 neural network
COCO Dataset Understanding: Work with the industry-standard dataset containing 80 common object categories
Object Tracking Algorithms: Develop control logic that uses bounding box coordinates to follow detected objects
Visual Servo Control: Implement proportional control systems that adjust robot motion based on visual feedback
Multi-Object Filtering: Learn techniques for selecting and tracking specific target objects from multiple detections
Real-Time Performance Optimization: Balance detection accuracy with inference speed for smooth robot behavior

💡 Why This Matters

Object detection and following represents a cornerstone capability for intelligent autonomous systems. Service robots use these techniques to follow humans in healthcare and hospitality settings. Warehouse automation systems track inventory items and navigate alongside workers. Agricultural robots identify and follow crop rows for automated harvesting. Autonomous vehicles detect and track pedestrians, other vehicles, and traffic signs for safe navigation. This laboratory bridges the gap between computer vision theory and practical robotic applications, demonstrating how neural networks enable robots to perceive and interact with their environment in meaningful ways. The skills you develop here—combining perception, decision-making, and actuation—form the foundation for virtually all autonomous robot behaviors.

Lab Structure

This laboratory consists of 4 progressive parts, each building upon previous concepts:

Intro Notebook: Live Camera Streaming with YOLO26 — a standalone warm-up exercise to verify YOLO26 detection on the CSI camera before adding robot motion
Part 1: YOLO26 Setup and Basic Object Detection — load the pre-trained model, understand COCO classes, and perform detection on live JetBot camera feeds
Part 2: Object Tracking and Filtering — implement algorithms to select specific objects from detections and track their positions over time
Part 3: Autonomous Object Following — develop complete behavior control that combines detection, tracking, and motor commands to follow target objects while avoiding obstacles

🎯

Learning Objectives

↑ Go Up

By the end of this laboratory session, you will be able to:

Implement YOLO26 object detection on edge devices using the Ultralytics framework for real-time computer vision applications.
Understand the COCO dataset structure and work with the 80 object classes commonly used in computer vision benchmarks.
Process detection outputs including bounding boxes, confidence scores, and class labels to extract actionable information.
Design object tracking algorithms that select and follow specific objects from multiple detections in complex environments.
Implement visual servo control using proportional control to adjust robot motion based on object position in the camera frame.
Integrate perception and action by combining YOLO26 detections with motor control for autonomous object following behavior.
Optimize real-time performance by balancing detection accuracy, inference speed, and control loop frequency for smooth robot operation.
Apply autonomous tracking principles to practical scenarios including human following, package delivery, and interactive robotics applications.

📚

Background

↑ Go Up

YOLO26: The Latest Generation of Real-Time Object Detection

YOLO (You Only Look Once) represents a revolutionary approach to object detection that treats detection as a single regression problem, directly predicting bounding boxes and class probabilities from full images in one evaluation. Unlike traditional methods that apply classifiers to different regions of an image, YOLO looks at the entire image during test time, enabling it to make predictions with remarkable speed while maintaining high accuracy.

YOLO26 is the latest generation in the YOLO family, continuing the tradition of improvements in accuracy, speed, and ease of deployment. It builds on its predecessors by offering a refined architecture optimized for edge hardware such as the NVIDIA Jetson Orin Nano. The model is available in multiple sizes — nano (n), small (s), medium (m), large (l), and extra-large (x) — allowing developers to choose the right trade-off between inference speed and detection accuracy. In this lab we use the nano variant (yolo26n.pt) for maximum real-time performance.

The COCO Dataset and Object Classes

The Common Objects in Context (COCO) dataset is a large-scale object detection, segmentation, and captioning dataset developed by Microsoft. It contains over 200,000 labeled images with 80 object categories that represent common objects found in everyday scenes. These categories include people, vehicles (car, motorcycle, airplane, bus, train, truck), animals (cat, dog, bird, horse, elephant), furniture, electronics, food items, and many others.

The YOLO26 model you'll use has been pre-trained on the COCO dataset, meaning it can already recognize these 80 categories without requiring any training on your part. Each detected object comes with a bounding box (defining its location in the image), a class label (what type of object it is), and a confidence score (how certain the model is about the detection). This pre-trained capability allows you to immediately apply sophisticated object detection to your robotic applications.

Object Following with Visual Servo Control

Object following combines computer vision (detecting and locating objects) with control systems (adjusting robot behavior based on observations). The basic approach uses visual servo control, where the position of the detected object in the camera frame serves as feedback for a control loop. When an object appears in the left side of the frame, the robot turns left; when it appears on the right, the robot turns right; when it's centered but small (far away), the robot moves forward.

A simple proportional controller calculates motor commands based on the horizontal position of the object's bounding box center. The error term is the difference between the object's x-coordinate and the center of the image. This error is multiplied by a gain constant to determine the turning speed, while forward speed depends on the object's distance (estimated from bounding box size). Proper tuning of these control parameters ensures smooth, stable following behavior without oscillation or overshoot.

Understanding Detection Outputs

When YOLO26 processes an image, it returns a set of detections, where each detection consists of three key pieces of information. The bounding box defines the rectangular region (x, y, width, height) that encloses the detected object. The class label identifies which of the 80 COCO categories the object belongs to, such as "person," "car," or "sports ball." The confidence score is a probability value between 0 and 1 indicating how certain the model is about this detection.

Understanding these outputs is crucial for implementing object following. The bounding box center coordinates tell us where the object is located in the frame, which drives our steering decisions. The class label allows us to filter detections and follow only specific types of objects. The confidence score helps us ignore uncertain detections that might be false positives. By setting an appropriate confidence threshold (typically 0.5 or higher), we can ensure our robot only responds to reliable detections.

Challenges in Real-Time Object Following

Several challenges arise when implementing object following on mobile robots. Detection latency introduces delays between when an object moves and when the robot responds. Multiple objects in the scene require filtering logic to select and track the intended target. Occlusions can cause temporary loss of detection, requiring prediction or memory of the object's last known position. Lighting changes, motion blur, and perspective distortions affect detection reliability.

Temporal Filtering: To smooth out noisy detections and prevent erratic robot behavior, we apply filtering techniques that average position estimates over multiple frames. This reduces the impact of momentary detection errors or jitter in bounding box coordinates.

Target Selection: When multiple objects of the same class appear in view, we need logic to consistently track the intended target. Common approaches include tracking the largest object, the one closest to the image center, or the one nearest to the previous detection position.

Handling Occlusions: Brief occlusions are common in real-world scenarios. Robust systems maintain memory of the target's last known position and velocity, allowing the robot to continue tracking even during short detection gaps.

Control System Design for Following

The control system for object following typically uses a proportional controller where motor commands are calculated based on the object's position in the camera frame. The horizontal error (difference between object center and image center) determines steering: negative error means the object is left of center, so we turn left; positive error means turn right. The magnitude of the error determines how aggressively we turn.

Forward speed is controlled based on the object's apparent size (bounding box area or height). When the object appears small (far away), we increase forward speed to approach it. When it appears large (close), we slow down or stop to maintain a safe following distance. This creates a natural "approach and maintain distance" behavior similar to adaptive cruise control in vehicles.

Tuning the control gains (proportional constants) is critical for stable behavior. Too low, and the robot responds sluggishly; too high, and it oscillates or overshoots. Finding the right balance requires testing with your specific hardware and target objects to achieve smooth, stable tracking.

Why YOLO26 for Edge Devices?

YOLO26 is particularly well-suited for edge devices like the Jetson Orin Nano because of its optimized architecture that achieves real-time performance with limited computational resources. The model's design prioritizes inference speed while maintaining competitive accuracy, making it possible to process camera frames at 20–30 FPS even on embedded hardware. This real-time capability is essential for responsive robot control, where delays between perception and action can lead to poor tracking performance or safety issues. The availability of multiple model sizes (nano to extra-large) allows developers to choose the right balance for their specific application requirements. As a drop-in replacement for previous YOLO versions, YOLO26 requires no changes to existing pipelines — simply swap the model file name.

📝

Pre-lab Preparation

↑ Go Up

Before starting the laboratory exercises, complete the following knowledge assessment quiz. These questions test your understanding of object detection concepts, YOLO26 architecture, visual servo control, and autonomous tracking principles.

📝 Pre-Lab Knowledge Assessment

Instructions: Answer the following 10 questions to assess your readiness for the object following laboratory. Click on your chosen answer to check if it's correct.

Question 1: What is the primary advantage of the YOLO approach to object detection compared to traditional region-based methods?

A) YOLO provides higher accuracy on all object classes
B) YOLO processes the entire image in a single forward pass, enabling real-time performance
C) YOLO requires less training data than other methods
D) YOLO can only detect one object per image

Question 2: How many object classes can the YOLO26 model pre-trained on the COCO dataset detect?

A) 20 classes
B) 40 classes
C) 80 classes
D) 1000 classes

Question 3: What information does a bounding box provide for a detected object?

A) Only the object's class label
B) Only the object's confidence score
C) The rectangular region (x, y, width, height) enclosing the object
D) The object's 3D position in space

Question 4: In visual servo control for object following, what does the error term typically represent?

A) The detection confidence score
B) The difference between the object's position and the desired position (usually image center)
C) The number of objects detected in the frame
D) The battery level of the robot

Question 5: What is model fusion in the context of YOLO26, as used in model.fuse()?

A) Combining multiple YOLO26 models to build a larger ensemble detection system
B) Optimizing the model by merging certain layers to reduce inference time
C) Mixing different datasets during training
D) Connecting the model to the camera

Question 6: If a detected object appears on the left side of the camera frame, how should the robot respond to center the object?

A) Turn left to move the object toward the frame center
B) Turn right to move the object toward the frame center
C) Move forward without turning
D) Move backward without turning

Question 7: What does a confidence score of 0.85 on a detected object indicate?

A) The object occupies 85% of the image
B) The model is 85% confident that the detection is correct
C) The object is 85 pixels wide
D) The inference took 85 milliseconds

Question 8: Why is temporal filtering (averaging detections over multiple frames) useful in object tracking?

A) It increases the detection confidence scores
B) It reduces noise and prevents erratic robot behavior from momentary detection errors
C) It speeds up the neural network inference
D) It eliminates the need for bounding boxes

Question 9: What challenge arises when multiple objects of the same class appear in the camera frame?

A) The camera stops working
B) YOLO26 can only detect one object at a time due to single-pass processing
C) The robot needs logic to select and consistently track the intended target object
D) The inference time becomes infinite

Question 10: In the context of object following, what does "visual servo control" mean?

A) Using mechanical sensors to control robot motion
B) Using visual feedback from cameras to adjust robot behavior in real-time
C) Controlling the camera zoom and focus settings
D) Programming the robot's display screen

✅ Pre-Lab Checklist

Before coming to lab, ensure you have:

✓ Completed all 10 quiz questions above
✓ Reviewed Week 10 materials on basic motion and collision avoidance
✓ Familiarized yourself with Python image processing basics
✓ Understood proportional control concepts from control systems theory
✓ Read about the COCO dataset and its 80 object classes

⚙️

Lab Procedure

↑ Go Up

This laboratory consists of an introductory notebook followed by three progressive parts that build your understanding of object detection and autonomous following. Each exercise uses YOLO26 for real-time computer vision on the JetBot platform. Work through each part sequentially, testing your implementations thoroughly before proceeding to the next.

🎥 Intro Notebook: Live Camera Streaming with YOLO26

Before operating the JetBot, you will first run a standalone warm-up notebook that demonstrates YOLO26 object detection on a live CSI camera feed — with no robot motion involved. This lets you verify the model is working correctly, get comfortable reading detection outputs, and observe real-time FPS performance on the Jetson hardware.

🎯 Key Learning Points

Load YOLO26n and apply model.fuse() for optimized inference
Initialize the Jetson CSI camera using the CSICamera class at 640×480
Use results[0].plot() to auto-annotate frames with all detected objects
Measure and display live FPS using time.perf_counter()
Use camera.observe() and camera.unobserve() to start and stop the detection loop
Understand the event-driven callback pattern before applying it to the JetBot

Implementation Steps:

Import Libraries: Import YOLO from ultralytics, cv2, CSICamera from jetcam, and time
Load YOLO26n: Run model = YOLO("yolo26n.pt") then model.fuse() to optimize for edge inference
Initialize Camera: Create a CSICamera instance at 640×480, 30 FPS, device 0
Capture Test Frame: Call camera.read(), display the frame in a Jupyter widget to confirm camera is working
Define update_image(): Inside the callback, calculate FPS, run model(image, conf=0.5, verbose=False), annotate with results[0].plot(), and overlay FPS text using cv2.putText()
Start Stream: Set camera.running = True and register the callback with camera.observe(update_image, names='value')
Stop Stream: Call camera.unobserve(update_image, names='value') to pause or end detection

⚠️ Key Difference from the JetBot Notebook:

This notebook uses CSICamera (640×480) — the JetBot notebook uses Camera (640×640)
results[0].plot() auto-draws all detections — the JetBot notebook manually calls cv2.rectangle() to control which box is highlighted
No robot motors are used here — this is a pure vision exercise
There is no robot.stop() needed; simply call camera.unobserve() to finish

📄 Open Intro Notebook

📦 Part 1: YOLO26 Setup and Basic Object Detection

In this first part, you'll set up the YOLO26 environment, load the pre-trained model, and implement basic object detection on the live JetBot camera feed. You'll learn how to process detection results including bounding boxes, class labels, and confidence scores.

🎯 Key Learning Points

Load and initialize the YOLO26n (nano) pre-trained model
Apply model fusion optimization for faster inference
Capture and process camera frames from the JetBot
Extract detection information (bounding boxes, classes, confidence)
Visualize detections with annotated bounding boxes
Understand COCO class names and indices

Implementation Steps:

Import Required Libraries: Import the YOLO class from ultralytics, OpenCV for image processing, and necessary JetBot modules for camera control
Load Pre-trained Model: Initialize YOLO26n using model = YOLO("yolo26n.pt") and apply model fusion with model.fuse()
Load COCO Class Names: Read the coco.names file to map class indices to human-readable labels
Initialize Camera: Set up the JetBot camera with appropriate resolution and frame rate
Implement Detection Loop: Capture frames, run inference, and extract detection results
Parse Detection Outputs: Extract bounding box coordinates, confidence scores, and class labels from model predictions
Visualize Results: Draw bounding boxes and labels on frames, display detection information
Measure Performance: Monitor inference time (preprocessing, inference, postprocessing) and frame rate

⚠️ Important Notes:

The YOLO26n model (yolo26n.pt) is already installed on your Jetson Orin Nano
Typical inference time is 30-50ms per frame on the Jetson platform
Confidence threshold of 0.5 is recommended for reliable detections
Test with various objects from the COCO dataset classes

📄 Open Exercise Notebook

🎯 Part 2: Object Tracking and Target Selection

Building on basic detection, this part focuses on tracking specific objects over time and implementing filtering logic to select target objects from multiple detections. You'll develop algorithms that maintain focus on a single object even when multiple candidates are present.

🎯 Key Learning Points

Filter detections by class type (e.g., only track "person" class)
Calculate bounding box center coordinates for tracking
Implement target selection based on size, position, or proximity
Maintain tracking consistency across consecutive frames
Handle occlusions and temporary detection loss
Apply temporal filtering to smooth position estimates

Implementation Steps:

Class-Based Filtering: Select detections matching specific COCO classes (e.g., filter for "person", "sports ball", or "cup")
Calculate Bounding Box Properties: Compute center coordinates (cx, cy), width, height, and area for each detection
Implement Selection Logic: Choose target based on criteria such as largest object, closest to center, or nearest to previous position
Track Target Over Time: Store target information and compare with new detections to maintain consistency
Handle Multiple Detections: Implement logic for when multiple objects of target class appear
Smooth Position Estimates: Apply moving average or low-pass filter to reduce jitter
Visualize Tracking: Highlight selected target with distinct color, show target path or trajectory
Test Edge Cases: Verify behavior when target enters/exits frame or becomes temporarily occluded

                        💡 Design Consideration: For robust tracking, consider using a combination of class matching, position proximity (IoU with previous detection), and confidence scores to maintain target identity across frames. A simple nearest-neighbor approach often works well for single-object scenarios.
                    

🤖 Part 3: Autonomous Object Following Behavior

In this final part, you'll integrate object detection and tracking with motor control to create complete autonomous following behavior. The robot will use visual servo control to track and follow target objects while maintaining appropriate distance and avoiding obstacles.

🎯 Key Learning Points

Implement proportional control for steering based on object position
Calculate forward speed based on object distance (bounding box size)
Combine detection, tracking, and motor control in a unified control loop
Handle scenarios when no target is detected (stop or search behavior)
Integrate collision avoidance from Week 10 with object following
Tune control parameters for smooth, stable following behavior

Implementation Steps:

Define Control Parameters: Set gain constants for steering (Kp_turn) and forward motion (Kp_forward), define minimum/maximum speeds
Calculate Steering Error: Compute horizontal error as difference between object center x-coordinate and image center
Implement Proportional Steering: Calculate turn speed as turn_speed = Kp_turn * error_x, clip to safe limits
Estimate Object Distance: Use bounding box height or area as proxy for distance, normalize to 0-1 range
Control Forward Motion: Set forward speed inversely proportional to estimated distance (slow when close, faster when far)
Handle No Detection: Implement stop behavior or slow rotation to search for target when no detection exists
Integrate Collision Avoidance: Add safety layer that overrides following commands if obstacles detected
Tune and Test: Adjust control gains through testing, ensure smooth following without oscillation

⚠️ Safety Considerations:

Always implement maximum speed limits (recommend 0.3-0.5 for safe operation)
Include emergency stop functionality (press key or button to immediately stop motors)
Maintain minimum distance from target (stop if bounding box exceeds size threshold)
Test in open area with clear surroundings before attempting complex environments
Keep manual control available to override autonomous behavior if needed

                        🔧 Recommended Control Parameters:
                        Kp_turn: 0.001 - 0.003 (adjust based on responsiveness needs)
Kp_forward: 0.3 - 0.5 (tune for desired following distance)
Max speed: 0.3 - 0.5 (balance between responsiveness and safety)
Stop threshold: Bounding box area > 40% of frame (adjust for comfort distance)
Temporal filter alpha: 0.3 - 0.5 (for position smoothing)

                    

✅ Testing Checklist

Verify your object following system works correctly:

✓ Robot correctly identifies and tracks target object class
✓ Robot turns left when object is on left side of frame
✓ Robot turns right when object is on right side of frame
✓ Robot moves forward when object is centered but far away
✓ Robot slows or stops when object is close (large bounding box)
✓ Robot stops safely when target exits frame
✓ Robot maintains smooth motion without excessive oscillation
✓ Emergency stop functionality works instantly
✓ Collision avoidance integrates properly with following behavior

🔧

Lab Materials

↑ Go Up

🤖 Hardware Platform

This laboratory uses the JetBot platform that you assembled and configured in Week 1. All necessary hardware components are already integrated and ready for use:

NVIDIA Jetson Orin Nano: Provides GPU acceleration for YOLO26 inference (8-core ARM CPU, NVIDIA Ampere GPU with 1024 CUDA cores, 8GB unified memory)
IMX219 8MP Camera: Captures 640×480 or 1280×720 video feeds for object detection (200° FOV)
Motor System: Differential drive with SparkFun QWIIC Motor Driver for autonomous navigation
Power System: Charmcast 10400mAh power bank providing adequate runtime for extended testing
OLED Display: Shows system status, detection counts, and performance metrics

💻 Software Environment

All required software packages have been pre-installed and configured by the lab technician on your Jetson system. Your environment includes:

JetPack SDK: NVIDIA's comprehensive AI development toolkit with optimized libraries
PyTorch: Deep learning framework with CUDA support for GPU acceleration
Ultralytics YOLO26: Latest YOLO generation with pre-trained models and utilities
OpenCV: Computer vision library for image processing and visualization
JetBot Python API: Robot control interface for motors, camera, and sensors
Jupyter Lab: Interactive notebook environment for development and testing

Note: No software installation is required. Simply access the Jupyter Lab interface through your web browser at the IP address displayed on your JetBot's OLED screen.

📁 Required Files and Datasets

The following files are provided in your JetBot's working directory:

yolo26n.pt — Pre-trained YOLO26 nano model weights
coco.names — Text file containing 80 COCO dataset class names
0_Jetson_YOLO_Object_Detection.ipynb — Intro notebook: live camera streaming and YOLO26 detection (no robot motion)
live_demo_yolov8.ipynb — JetBot notebook: full object following with motor control

🎯 Testing Environment Requirements

For safe and effective testing of your object following implementation, ensure you have:

Open Space: At least 3m × 3m clear area free of obstacles for initial testing
Good Lighting: Well-lit environment (avoid direct sunlight or very low light)
Target Objects: Common objects from COCO classes (e.g., sports ball, cup, cell phone, person)
Safety Barrier: Designated testing area marked to prevent unintended collisions
Emergency Stop: Accessible wireless connection or physical access for immediate shutdown

⚠️ Important Reminders:

Always test autonomous behaviors in controlled environments first
Keep emergency stop readily available during all autonomous operation
Start with low speed settings and gradually increase after verifying safe operation
Monitor battery level - performance degrades significantly below 20%
Avoid testing near stairs, drops, or areas with valuable/fragile items

📖

References

↑ Go Up

Official Documentation

Ultralytics YOLO Documentation: Complete Documentation and Tutorials
Comprehensive guide to YOLO architecture, training, inference, and deployment — covers all versions including YOLO26

YOLO Detection Task: Object Detection Guide
Detailed documentation on using Ultralytics YOLO for object detection applications

COCO Dataset: Common Objects in Context
Official COCO dataset website with class descriptions and benchmarks

COCO Class Names: Complete List of 80 Object Classes
YAML configuration file listing all COCO dataset categories and indices

NVIDIA JetBot: Official GitHub Repository
Open-source JetBot project with examples and API documentation

JetBot Orin Nano (UAEU): Course Repository – Notebooks, Setup Guides, and Resources
UAEU ELEC 395 course repository with JetBot Orin Nano notebooks and lab materials

Research Papers and Technical Articles

You Only Look Once: Unified, Real-Time Object Detection
Redmon et al., 2016 - Original YOLO paper introducing the single-shot detection paradigm

YOLO9000: Better, Faster, Stronger
Redmon and Farhadi, 2017 - YOLOv2 improvements and multi-scale predictions

YOLO: Real-Time Object Detection
Joseph Redmon's comprehensive YOLO project page with examples and demos

Microsoft COCO: Common Objects in Context
Lin et al., 2014 - Original COCO dataset paper describing the benchmark and evaluation metrics

Tutorials and Learning Resources

PyTorch Vision Documentation
Official documentation for torchvision including pre-trained models and transforms

OpenCV University - Computer Vision Courses
Free educational resources for learning computer vision fundamentals

NVIDIA Jetson Tutorials
Official NVIDIA tutorials for AI on Jetson platforms including optimization techniques

Lab Report

↑ Go Up

Submit a comprehensive laboratory report documenting your object detection and following implementation. Your report should demonstrate understanding of YOLO26 architecture, visual servo control principles, and practical autonomous robot behavior. Include code snippets, screenshots, and analysis of your system's performance.

📋 Report Structure (100 points)

Your lab report must include the following sections:

1. Title Page & Formatting (5 points)

Lab title, your name, student ID, date, course name, and instructor name
Professional formatting with clear headers and page numbers

2. Objectives (10 points)

List all learning objectives addressed in this laboratory
Briefly explain the significance of object detection for autonomous robotics

3. Procedure & Results (50 points)

For each of the 3 parts, include:

Intro Notebook: Screenshot of YOLO26 live detections from the CSI camera with FPS counter visible
Part 1: Screenshots of YOLO26 detections on the JetBot camera with bounding boxes, class labels, confidence scores, and performance metrics
Part 2: Code snippets showing tracking algorithms, visualizations of object trajectories, and filtering results
Part 3: Complete control algorithm code, motor command graphs, video stills of following behavior, and performance analysis
Clear explanations of what each part demonstrates

4. Discussion (20 points)

Compare YOLO26 performance on Jetson vs. cloud-based alternatives, and discuss any observed improvements over previous YOLO versions
Analyze factors affecting detection reliability (lighting, distance, occlusion)
Discuss your control parameter tuning process and rationale
Evaluate integration of collision avoidance with following behavior
Support all statements with evidence from your experiments

5. Challenges & Solutions (10 points)

Describe technical challenges encountered (detection errors, control oscillation, etc.)
Explain your debugging process and solutions implemented
Reflect on lessons learned about real-time computer vision systems

6. Conclusion (5 points)

Summarize key learnings about object detection and autonomous tracking
Reflect on challenges of integrating perception and control
Discuss how these techniques apply to advanced robotics applications

📋 Submission Checklist

Before submitting, ensure you have:

✓ Completed all 3 lab parts with working implementations
✓ Included clear screenshots of detections with bounding boxes and labels
✓ Documented control algorithm with code and parameter values
✓ Provided performance analysis and metrics
✓ Added video stills or sequence showing object following behavior
✓ Included thoughtful discussion of results and improvements
✓ Described challenges and debugging process
✓ Formatted report professionally with clear headers and page numbers
✓ Proofread for grammar, spelling, and technical accuracy
✓ Verified all code is properly commented and explained

📤 Submission Format

File Format: Submit report as PDF document (required)
Code Files: Include Jupyter notebooks (.ipynb) in a separate ZIP file
File Naming Convention:
- Report: Week11_[YourLastName]_[StudentID].pdf
- Code: Week11_[YourLastName]_[StudentID]_Code.zip
- Example: Week11_Ahmed_202012345.pdf
Submission Method: Upload to University LMS (Blackboard/Moodle)
File Size Limit: Maximum 50MB total
- If exceeded, compress images or use PDF compression tools
- Ensure PDF is searchable text, not scanned images
Required Components:
- 1. Main PDF lab report
- 2. ZIP file containing all Jupyter notebooks with outputs
- 3. Any modified helper files (if applicable)

Important:

Ensure PDF is searchable and not password-protected
All code must be properly commented and executable
Include all necessary imports and dependencies
Test that your notebooks run completely from top to bottom

📊 Grading Rubric

Component	Points	Criteria
Title Page & Formatting	5	Complete, professional
Objectives	10	Clear, comprehensive
Procedure & Results	50	All parts complete, correct code, proper outputs
Discussion	20	Thoughtful analysis, supported by results
Challenges & Solutions	10	Detailed problem-solving process
Conclusion	5	Reflective, insightful
Total	100

Grading Notes:

All code must execute without errors for full credit
Screenshots must be clear, properly labeled, and referenced
All discussion questions must be answered with supporting evidence
Late penalty: 10% per day (up to 3 days)
Plagiarism will result in zero credit

Laboratory Overview

What You'll Learn

💡 Why This Matters

Lab Structure

Learning Objectives

Background

YOLO26: The Latest Generation of Real-Time Object Detection

The COCO Dataset and Object Classes

Object Following with Visual Servo Control

Understanding Detection Outputs

Challenges in Real-Time Object Following

Control System Design for Following

Why YOLO26 for Edge Devices?

Pre-lab Preparation

📝 Pre-Lab Knowledge Assessment

Question 1: What is the primary advantage of the YOLO approach to object detection compared to traditional region-based methods?

Question 2: How many object classes can the YOLO26 model pre-trained on the COCO dataset detect?

Question 3: What information does a bounding box provide for a detected object?

Question 4: In visual servo control for object following, what does the error term typically represent?

Question 5: What is model fusion in the context of YOLO26, as used in model.fuse()?

Question 6: If a detected object appears on the left side of the camera frame, how should the robot respond to center the object?

Question 7: What does a confidence score of 0.85 on a detected object indicate?

Question 8: Why is temporal filtering (averaging detections over multiple frames) useful in object tracking?

Question 9: What challenge arises when multiple objects of the same class appear in the camera frame?

Question 10: In the context of object following, what does "visual servo control" mean?

✅ Pre-Lab Checklist

Lab Procedure

🎥 Intro Notebook: Live Camera Streaming with YOLO26

🎯 Key Learning Points

Implementation Steps:

📦 Part 1: YOLO26 Setup and Basic Object Detection

🎯 Key Learning Points

Implementation Steps:

🎯 Part 2: Object Tracking and Target Selection

🎯 Key Learning Points

Implementation Steps:

🤖 Part 3: Autonomous Object Following Behavior

🎯 Key Learning Points

Implementation Steps:

✅ Testing Checklist

Lab Materials

🤖 Hardware Platform

💻 Software Environment

📁 Required Files and Datasets

🎯 Testing Environment Requirements

References

Official Documentation

Research Papers and Technical Articles

Tutorials and Learning Resources

Recommended Reading

Lab Report

📋 Report Structure (100 points)

1. Title Page & Formatting (5 points)

2. Objectives (10 points)

3. Procedure & Results (50 points)

4. Discussion (20 points)

5. Challenges & Solutions (10 points)

6. Conclusion (5 points)

📋 Submission Checklist

📤 Submission Format

📊 Grading Rubric

Grading Notes: