https://www.deeplearningwizard.com/deep_learning/boosting_models_pytorch/optimizers/ Optimization Algorithms - Deep Learning Wizard Optimization Algorithms Introduction to Gradient-descent Optimizers Model Recap: 1 Hidden Layer Feedforward Neural Network (ReLU Activation) Steps Step 1: Load Dataset Step 2: Make Dataset Iterable Step 3: Create Model Class Step 4: Instantiate Model Class www.deepl..

https://www.deeplearningwizard.com/deep_learning/boosting_models_pytorch/lr_scheduling/ Learning Rate Scheduling - Deep Learning Wizard Learning Rate Scheduling Optimization Algorithm: Mini-batch Stochastic Gradient Descent (SGD) We will be using mini-batch gradient descent in all our examples here when scheduling our learning rate Combination of batch gradient descent & stochastic gradien www.d..

https://www.deeplearningwizard.com/deep_learning/boosting_models_pytorch/derivative_gradient_jacobian/ Derivative, Gradient and Jacobian - Deep Learning Wizard Derivative, Gradient and Jacobian Simplified Equation This is the simplified equation we have been using on how we update our parameters to reach good values (good local or global minima) \theta = \theta - \eta \cdot \nabla_\theta \theta:..

https://towardsdatascience.com/hyper-parameter-tuning-techniques-in-deep-learning-4dad592c63c8 Hyper-parameter Tuning Techniques in Deep Learning The process of setting the hyper-parameters requires expertise and extensive trial and error. There are no simple and easy ways to set… towardsdatascience.com Hyper-parameters 1) learning rate 2) momentum 3) batch size 4) weight decay Momentum이란? - 기존 ..

https://www.deeplearningwizard.com/deep_learning/boosting_models_pytorch/forwardpropagation_backpropagation_gradientdescent/ Forward- and Backward-propagation and Gradient Descent (From Scratch FNN Regression) - Deep Learning Wizard Forwardpropagation, Backpropagation and Gradient Descent with PyTorch Transiting to Backpropagation Let's go back to our simple FNN to put things in perspective Let ..

https://www.deeplearningwizard.com/deep_learning/practical_pytorch/pytorch_feedforward_neuralnetwork/ Feedforward Neural Networks (FNN) - Deep Learning Wizard Feedforward Neural Network with PyTorch About Feedforward Neural Network Logistic Regression Transition to Neural Networks Logistic Regression Review Define logistic regression model Import our relevant torch modules. import torch import t..