Neural Network Parameter Counter

How do you count parameters in a neural network?

The number of trainable parameters in a neural network determines its capacity, memory requirements, and computational cost. A Dense (fully connected) layer has (input_size × output_size) + output_size parameters (weights plus biases). A Conv2D layer has (kernel_h × kernel_w × input_channels × output_filters) + output_filters parameters. An LSTM layer has 4 × ((input_size + hidden_size) × hidden_size + hidden_size) parameters, accounting for the four gates (input, forget, output, cell). For reference, GPT-3 has 175 billion parameters, while BERT-base has 110 million. Model size in memory is approximately parameters × 4 bytes (float32) or parameters × 2 bytes (float16). Counting parameters helps compare model complexity, estimate training time, and determine hardware requirements for deployment.