Advanced technique in data analysis and statistical learning

This course provides a comprehensive introduction to Machine Learning, covering both theoretical foundations and practical applications. It introduces students to supervised and unsupervised learning, focusing on model evaluation, feature selection, clustering, decision trees, and regression techniques.
Students will develop hands-on expertise in building, tuning, and evaluating models while understanding the mathematical principles behind them. The course emphasizes best practices in data preprocessing, model selection, and performance optimization, equipping students with the necessary tools to apply Machine Learning in real-world scenarios across various industries.

Linear & Logistic Regression

• Simple vs. Multiple Linear Regression
• Assumptions (Gauss-Markov Theorem)
• Model Inference & Statistical Testing (t-tests, F-tests, Confidence Intervals)
• Logistic Regression for Classification
• Handling Categorical Variables (Dummy Encoding, Polynomial Regression)
• Practical Example: Predicting house prices with multiple regression

Decision Trees

• Tree-Based Model Structure
• Splitting Criteria (Gini Index, Entropy)
• Overfitting Prevention (Pre-Pruning, Post-Pruning)
• Computational Complexity Considerations
• Practical Example: Decision tree classification on real-world data

Model Evaluation & Feature Selection

• Understanding Bias-Variance Trade-off
• Cross-Validation techniques
• Evaluation Metrics (MAE, RMSE, R², AUC-ROC, Confusion Matrix)
• Feature Selection Techniques (Wrapper, Filter, Regularization)
• Hyperparameter Tuning (Grid Search, Bayesian Optimization, AutoML)
• Practical Example: Model selection and tuning using Python

Clustering Techniques

• Introduction to Unsupervised Learning
• Hierarchical Clustering (Agglomerative, Divisive)
• Partitioning Methods (K-Means, K-Medoids)
• Density-Based Clustering (DBSCAN, OPTICS)
• Gaussian Mixture Models with Expectation-Maximization
• Practical Example: Clustering customer data for segmentation

Deep Learning

This course provides an in-depth exploration of Deep Learning, covering fundamental concepts, state-of-the-art architectures, and practical applications. It introduces students to neural networks, optimization techniques, and specialized models such as CNNs, RNNs, and Transformers.

History and evolution of neural networks

• Why deep learning works today (Data, Compute, and Algorithm improvements)
• Introduction to Perceptron and activation functions (Sigmoid, ReLU, Tanh)
• Gradient Descent and Backpropagation
• Implementing a simple perceptron in Python
• Visualizing activation functions and their derivatives

Deep Neural Networks & Optimization

• Forward and Backward Propagation
• Cost functions (Cross-Entropy, MSE)
• Regularization techniques (L1, L2, Dropout)
• Hyperparameter tuning (Learning Rate, Batch Size, Epochs)
• Gradient Descent Variants: SGD, Momentum, RMSprop, Adam
• Implementing a basic multi-layer perceptron (MLP)
• Using dropout and batch normalization in training

Convolutional Neural Networks (CNNs)

• Why CNNs? Handling high-dimensional image data
• Filters, Kernels, and Convolution operations
• Pooling layers (Max Pooling, Average Pooling)
• Popular CNN Architectures (LeNet, AlexNet, VGG, ResNet)
• Implementing a CNN for image classification
• Training a CNN on MNIST/CIFAR dataset

Recurrent Neural Networks (RNNs) & LSTMs

• Sequence modeling and challenges (Vanishing gradient)
• Recurrent Neural Networks (RNN) basics
• Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU)
• Applications: Time series prediction, NLP
• Implementing a simple RNN
• Training an LSTM model for text generation

Transformers & Attention Mechanisms

• Attention Mechanism & Self-Attention
• Transformer architecture overview
• Pre-trained models: BERT, GPT, and Hugging Face models
• Fine-tuning a Transformer for a custom NLP task