Examples and Intuitions I — Neural Networks as Logical Gates

Example 1: Implementing the AND Operator

A simple example of applying neural networks is predicting:

$$x_1 \land x_2$$

The logical AND operator is true only when:

• $x_1 = 1$
• $x_2 = 1$

Otherwise, it is false.

---

Network Structure

Our small neural network looks like:

$$\begin{bmatrix} x_0 \\ x_1 \\ x_2 \end{bmatrix} \rightarrow g(z^{(2)}) \rightarrow h_\Theta(x)$$

Remember:

$$x_0 = 1$$

This is the bias unit.

---

Choosing the Weights

Let us define the weight matrix:

$$\Theta^{(1)} = \begin{bmatrix} -30 & 20 & 20 \end{bmatrix}$$

The hypothesis becomes:

$$h_\Theta(x) = g(-30 + 20x_1 + 20x_2)$$

---

Evaluating All Input Combinations

Case 1

$$x_1 = 0, \quad x_2 = 0$$ $$g(-30) \approx 0$$

---

Case 2

$$x_1 = 0, \quad x_2 = 1$$ $$g(-10) \approx 0$$

---

Case 3

$$x_1 = 1, \quad x_2 = 0$$ $$g(-10) \approx 0$$

---

Case 4

$$x_1 = 1, \quad x_2 = 1$$ $$g(10) \approx 1$$

---

Conclusion

With this choice of weights:

$$\Theta^{(1)} = \begin{bmatrix} -30 & 20 & 20 \end{bmatrix}$$

the neural network behaves exactly like an AND gate.

We constructed a fundamental logical operation using a logistic neuron.

---

Example 2: Implementing the OR Operator

The logical OR operator is true when:

• $x_1 = 1$, or
• $x_2 = 1$, or both

We can implement OR using a different set of weights:

$$\Theta^{(1)} = \begin{bmatrix} -10 & 20 & 20 \end{bmatrix}$$

The hypothesis becomes:

$$h_\Theta(x) = g(-10 + 20x_1 + 20x_2)$$

---

Evaluating OR

Case 1

$$x_1 = 0, \quad x_2 = 0$$ $$g(-10) \approx 0$$

---

Case 2

$$x_1 = 0, \quad x_2 = 1$$ $$g(10) \approx 1$$

---

Case 3

$$x_1 = 1, \quad x_2 = 0$$ $$g(10) \approx 1$$

---

Case 4

$$x_1 = 1, \quad x_2 = 1$$ $$g(30) \approx 1$$

---

Key Intuition

A single logistic neuron can simulate logical gates.

By adjusting:

• Bias (threshold)
• Weights (importance of inputs)

we can model:

• AND
• OR
• NAND
• NOR

Neural networks are powerful because stacking these simple units allows us to represent much more complex functions.