A neural network transforms numbers through weighted sums and nonlinear functions. This simulation exposes a network with two inputs, three hidden neurons, and one output. Adjust the inputs to inspect the calculation, choose a target, and run a training step to change the weights.

The forward pass

Each neuron combines its inputs with weights and a bias. For one layer, the operation is:

y=σ(W⋅x+b).y = \sigma(W \cdot x + b).

Here we distinguish the hidden activation ϕ\phi from the sigmoid output:

h=ϕ(W1x+b1),y^=σ(W2h+b2).h = \phi(W_1x+b_1), \qquad \hat y = \sigma(W_2h+b_2).

The forward pass evaluates these operations in order. Changing an input changes the sums, activations, and prediction while keeping the weights fixed. Training is a separate operation.

Neural network · 2 → 3 → 1One example, one training step
InputsHidden layerOutput · sigmoidw = 0.8; a = 0.5; w × a = 0.4w = -0.4; a = 0.25; w × a = -0.1w = -0.3; a = 0.5; w × a = -0.15w = 0.9; a = 0.25; w × a = 0.225w = 0.6; a = 0.5; w × a = 0.3w = 0.5; a = 0.25; w × a = 0.125w = 0.7; a = 0.610639233949222; w × a = 0.4274474637644554w = -0.6; a = 0.5683199834782481; w × a = -0.34099199008694886w = 0.5; a = 0.6283161882953663; w × a = 0.31415809414768314x₁0.5x₂0.25h10.6106h20.5683h30.6283ŷ0.5746

● Positive contribution ● Negative contribution

Edge thickness shows |weight × source activation|. Node intensity shows activation magnitude, saturating at 1. The output always uses sigmoid. On small screens, scroll the diagram horizontally.

Prediction ŷ
0.5746
Loss ½(ŷ − t)²
0.0254
Training steps
0

Adjust the inputs, choose a target, and train on this example.

Inspect calculations and weights
NeuronzActivationWeightsBias
h1
h2
h3
ŷ

Sigmoid and ReLU

Sigmoid maps any real number to a value between zero and one. ReLU preserves positive values and sets negative values to zero:

σ(z)=11+e−z,ReLU⁡(z)=max⁡(0,z).\sigma(z)=\frac{1}{1+e^{-z}}, \qquad \operatorname{ReLU}(z)=\max(0,z).

The selector changes the three hidden neurons. The output always uses sigmoid. Node intensity represents activation, while connections show the magnitude and sign of the contribution w aw\,a, not a probability or causal explanation. Changing an input’s sign can change the contribution’s sign even if its weight remains unchanged.

One gradient-descent step

For a target tt, the loss on this example is:

L=12(y^−t)2,θk+1=θk−η ∇θL.\mathcal L=\frac12(\hat y-t)^2, \qquad \theta_{k+1}=\theta_k-\eta\,\nabla_{\theta}\mathcal L.

The button computes derivatives using backpropagation and updates both layers’ weights and biases simultaneously. If the chosen learning rate fails to reduce the loss, it tries a smaller step. The interface reports the loss before and after the update and the effective learning rate. ReLU uses a zero derivative at the origin; an inactive neuron can receive a zero gradient.

Each click trains on the visible example, not on a dataset. A lower loss here does not demonstrate generalization. Reset restores the initial inputs, weights, and target so the experiment can be repeated.