Below is a comprehensive breakdown of topics from all weeks that will be assessed in the final examination:
Python for Real Hardware - Jetson & Raspberry Pi
Key Topics:
- NVIDIA Jetson Orin Nano specifications and capabilities (8-core ARM CPU, NVIDIA Ampere GPU, 8GB memory)
- JetBot robot assembly, component identification, and system architecture
- Hardware interfaces: GPIO, I2C, SPI, UART protocols
- Power management and thermal considerations for embedded systems
- Basic Linux commands for embedded systems administration
- Raspberry Pi comparison and use cases
Fundamentals of Machine Learning and Artificial Intelligence
Key Topics:
- Introduction to artificial intelligence and machine learning concepts
- Supervised vs. unsupervised vs. reinforcement learning paradigms
- Gradient descent optimization algorithm and its variants (SGD, Adam, RMSprop)
- Loss functions: MSE, cross-entropy, binary cross-entropy
- Activation functions: ReLU, Sigmoid, Tanh, Softmax - properties and use cases
- Backpropagation and chain rule fundamentals
- Overfitting, underfitting, and regularization techniques
Neural Network Architectures - ANN, DNN, CNN
Key Topics:
- PyTorch tensors: creation, manipulation, and operations
- Artificial Neural Networks (ANN) structure and forward propagation
- Deep Neural Networks (DNN) with multiple hidden layers
- Convolutional Neural Networks (CNN) architecture and components
- Convolution operations, kernels/filters, stride, and padding
- Pooling layers: max pooling, average pooling
- Training neural networks in PyTorch: optimizer, loss, training loop
- Fashion-MNIST dataset and image classification
- Model evaluation: accuracy, precision, recall, F1-score
Data Representation and Image Datasets
Key Topics:
- MNIST handwritten digit dataset structure and applications
- CIFAR-10 dataset: 10 classes of color images
- IRIS dataset for classification tasks
- Image preprocessing: normalization, resizing, augmentation
- Data loading and batching in PyTorch (DataLoader, Dataset classes)
- Train/validation/test split strategies
- CNN architectures for image classification
- Transfer learning concepts
Python ML Libraries - PyTorch & TensorFlow
Key Topics:
- PyTorch framework fundamentals and tensor operations
- TensorFlow/Keras basics and comparison with PyTorch
- Scikit-learn for classical machine learning algorithms
- Model training, validation, and testing procedures
- Hyperparameter tuning strategies
- Model saving and loading (state_dict, checkpoints)
- GPU acceleration and CUDA operations
- Practical deep learning workflows
Sensor Interface and Data Acquisition
Key Topics:
- CSI (Camera Serial Interface) cameras and MIPI protocol
- IMX219 camera specifications and configuration
- IยฒC communication protocol and addressing
- QWIIC system for sensor connectivity
- BME280 environmental sensor (temperature, humidity, pressure)
- GStreamer for video capture and processing
- Camera parameters: resolution, frame rate, exposure, white balance
- Sensor data collection and processing for AI applications
Classification Algorithms
Key Topics:
- Support Vector Machines (SVM): linear and non-linear kernels
- Decision Trees: splitting criteria, pruning, depth control
- Random Forest: ensemble learning, bagging, feature importance
- K-Nearest Neighbors (KNN) algorithm
- Classification metrics: confusion matrix, accuracy, precision, recall
- Cross-validation techniques
- Feature engineering and selection
- Comparison of classical ML vs. deep learning for classification
Regression Techniques
Key Topics:
- Linear regression: simple and multiple regression
- Polynomial regression for non-linear relationships
- Least squares method and cost functions
- Regularization in regression: L1 (Lasso), L2 (Ridge)
- Regression metrics: MSE, RMSE, R-squared, MAE
- Feature scaling and normalization importance
- Neural network-based regression
- Prediction vs. interpolation vs. extrapolation
Autonomous Driving - Collision Avoidance
Key Topics:
- Differential drive robot control and motor commands
- Data collection for collision avoidance (blocked vs. free paths)
- Training CNNs for binary classification (collision detection)
- Real-time inference on edge devices
- Motor control integration with vision system
- Emergency stop mechanisms and safety protocols
- Model optimization for low-latency inference
- Testing and validation of autonomous behaviors
Object Detection and Following with YOLOv8
Key Topics:
- YOLO (You Only Look Once) architecture and evolution
- YOLOv8 implementation using Ultralytics framework
- COCO dataset and 80 object classes
- Bounding box predictions and confidence scores
- Object tracking algorithms and filtering
- Visual servo control for object following
- Proportional control for steering based on object position
- Real-time object detection performance optimization
Autonomous Road Following
Key Topics:
- Regression-based road following vs. classification-based collision avoidance
- Data collection: interactive labeling of road center coordinates
- ResNet-18 architecture for regression tasks
- Training neural networks to predict continuous steering coordinates
- Coordinate-to-motor-command transformation
- Smooth control and trajectory planning
- Combining multiple AI models (collision avoidance + road following)
- Real-world deployment and testing strategies
Large Language Models (LLMs) and Edge Deployment
Key Topics:
- Introduction to Large Language Models (LLMs) and transformer architecture
- Ollama framework for running LLMs locally
- Edge AI deployment considerations: memory, compute, latency
- Quantization techniques for model compression
- Running LLMs on Jetson Orin Nano
- Prompt engineering and inference optimization
- Applications of edge LLMs in robotics
- Privacy and security benefits of on-device AI