Simulation: Life as a Survival Optimization Problem
As someone who came to the Machine Learning world from a Medical background, I couldn’t help not relating being stuck at a Local Maximum to other life situations. So I have decided to make a simulation project that helps to visualize this problem from a biological and also a political perspective where liberals and conservatives compete to reach the global maximum (Links at the end).
The geographical representation is a very nice way to visualize and relate the topics. For instance, internal struggle and Existential threats can be seen as means to help in reaching the Global Maximum. Also, sexual reproduction can be seen as a way to share information between two Local Maxima which can help in reducing the search space.
One can even relate this problem to politics! For instance, you can think of a Conservative as someone with a small step changing-rate while a Liberal as someone with a large step adaptation-rate. Political systems like Anarchy can be seen as a situation where no one sticks to any Local Maxima whereas fascism can be seen as everyone sticks to a single Local Maximum.
We know that DNA based life replicates and reproduces exponentially but because of resource limitations it will hit a ceiling and due to the internal struggle it optimizes itself for surviving better, hence, the survival of the fittest is becoming the objective of life.
An illustrative video description of the simulations is here:
The purpose of this simulation is not to present the state-of-the-art algorithms but to bridge and link the terms that are used in the Machine Learning world and the ones that are used in the biological and the political world by making these toy simulations and easily digestible videos to both parties. I believe the Machine learning world can give so much to the other fields because life in essence is a survival optimization problem where everything is complicated by being stuck at a Local Maximum.
Here you can play with the simulation: https://simmer.io/@hunar/reaching-global-maxima
Simulation Code: https://github.com/hunar4321/Reaching-global-maximum
Important Note:
Some experts say that Local Maxima doesn’t matter in very high dimensional landscapes. This is true if the convergence speed doesn’t matter and also if all dimensions have equal weights. However, we know that is not the case, convergence speed always matters in a competitive world like ours as all Life forms are in a tough race for survival. Also, not all dimensions have the same impactful weight. Many dimensions can be ignored or they are already pruned or not accessible, therefore, the actual number of the plausible dimensions is much less than the available dimensions.
Related topics:
Other related Algorithms: Neat algorithm, Hill Climbing, Particle swarm optimization…,etc.
Other Optimization techniques that uses Calculus: Gradient Descent, Adam, Recursive Least Squares…, etc.
Other related terms: Local minimum & Global minimum (when minimizing the error)
Deep Learning techniques are good to avoid being stuck at local maximum as they use many layers and lots of data.
Other learning methods: Hebbian Learning, Winner takes it all (WTA), …etc. We at Brainxyz use: Predictive Hebbian Unified Neurons (PHUN)
