Neural Networks: Neurons, Activation, Structure

Biological Neurons

A biological neuron is the fundamental unit of the nervous system, responsible for transmitting information throughout the body. It consists of three main parts:

• Dendrites: These are branch-like structures that receive signals from other neurons and transmit them to the cell body.
• Cell Body (Soma): Contains the nucleus and other essential organelles responsible for processing information.
• Axon: A long, thread-like extension that carries nerve impulses away from the cell body to other neurons, muscles, or glands.

Neurons communicate using electrical and chemical signals through synapses, where neurotransmitters help in transmitting the signals. The brain contains billions of neurons that work together to perform cognitive functions, control bodily movements, and process sensory inputs.

Activation Functions Explained

An activation function in artificial neural networks (ANNs) is a mathematical function that determines whether a neuron should be activated or not. It introduces non-linearity to the network, allowing it to learn and perform complex tasks. Types of activation functions:

1. Step Function: Outputs 0 or 1 based on a threshold.
2. Sigmoid Function: Outputs values between 0 and 1, useful for probability-based models.
3. Tanh Function: Outputs values between -1 and 1, making it useful for models requiring negative values.
4. ReLU (Rectified Linear Unit): Outputs 0 for negative inputs and the same value for positive inputs, widely used in deep learning.
5. Leaky ReLU: Similar to ReLU but allows small gradients for negative inputs.
6. Softmax Function: Converts values into probabilities, useful for classification problems.

Choosing the right activation function impacts the learning efficiency and accuracy of the neural network.

Neural Networks & Characteristics

A neural network is a computational model inspired by the human brain that processes information in layers. It consists of interconnected artificial neurons that work together to recognize patterns, classify data, and make predictions.

Characteristics of Neural Networks

1. Layers: Neural networks are made up of input, hidden, and output layers.
2. Weights and Biases: Each connection between neurons has a weight, and biases adjust the activation.
3. Learning Process: Uses backpropagation and optimization algorithms like gradient descent.
4. Non-linearity: Activation functions introduce non-linearity to model complex relationships.
5. Generalization: Neural networks can learn from training data and apply knowledge to new data.
6. Parallel Processing: Many neurons work together simultaneously to improve computational efficiency.

Neural networks are used in fields like image recognition, speech processing, and medical diagnosis.

ANN as a Graph Representation

An Artificial Neural Network (ANN) can be represented as a directed graph where:

• Nodes (Vertices) represent neurons.
• Edges (Connections) represent the weights between neurons.
• Input Nodes: Represent raw data features.
• Hidden Nodes: Perform intermediate computations and transformations.
• Output Nodes: Provide final results after processing.