A collection of various gradient descent algorithms implemented in Python from scratch

[ICML 2021] The official PyTorch Implementations of Positive-Negative Momentum Optimizers.

NAG-GS: Nesterov Accelerated Gradients with Gauss-Siedel splitting

Intelligent Detection for RIS-Assisted MIMO Systems: A First-and-Second Momentum Approach

Overshoot: Taking advantage of future gradients in momentum-based stochastic optimization

In this project it is used a Machine Learning model based on a method called Extreme Learning, with the employment of L2-regularization. In particular, a comparison was carried out between: (A1) which is a variant of incremental extreme learning machine that is QRIELM and (A2) which is a standard momentum descent approach, applied to the ELM.

Using Matrix Factorization/Probabilistic Matrix Factorization to solve Recommendation。矩阵分解进行推荐系统算法。

Python code for Gradient Descent, Momentum, and Adam optimization methods. Train neural networks efficiently.

Simple Document Classification using Multi Class Logistic Regression & SVM Soft Margin from scratch

Lightweight neural network library written in ANSI-C supporting prediction and backpropagation for Convolutional- and Fully Connected neural networks

Generic L-layer 'straight in Python' fully connected Neural Network implementation using numpy.

This is an implementation of different optimization algorithms such as: - Gradient Descent (stochastic - mini-batch - batch) - Momentum - NAG - Adagrad - RMS-prop - BFGS - Adam Also, most of them are implemented in vectorized form for multi-variate problems

Numerical Optimization for Machine Learning & Data Science

EE456 2022 mini project implementation of two-moons problem using multi-layer-perceptron with back-propagation with analyzing performance of initializing methods and momentum rule

Machine Learning, Deep Learning Implementations

This repository provides implementations of numerical optimization algorithms for machine learning and deep learning. It includes clear explanations, mathematical formulas, Python code, and visualizations to help understand the behavior of each optimizer.

Comparsion of Machine Learning optimization algorithms with MNIST dataset

This repository contains a python implementation of Feed Forward Neural Network with Backpropagation, along with the example scripts for training the network to classify images from mnist and fashion_mnist datasets from keras.