Why even solvable problems are often unsolvable
--
Solvability is a very interesting concept in mathematics by which one can determine, using sophisticated methods of proof and analysis, whether a certain problem can be solved, be it an equation, system of equations, integral, etc. Solvability usually refers to whether a problem can have an “exact or closed-form solution”, but other times it can also refer to whether a problem’s suitability to be solved via numerical or approximate methods.
For instance, certain linear partial differential equations such as the heat equation or wave equation can be solved exactly under certain initial or boundary conditions, and usually in very simple geometries like rectangular or circular domains. However, as soon as you approach the problem on a non-standard domain, or introduce more complicated boundary conditions, the differential equation becomes unsolvable unless you employ some numerical scheme.
This is precisely the issue one faces in the real world. It is easy to fall into the trap of thinking that because a certain mathematical problem can be solved numerically, that there are no computational challenges associated with it. “Oh well, the heat equation is easy to solve, so why do we need to spend all this time doing design and engineering simulation of heat transfer?” is the type of question one might ask while not knowing how a problem scales with the size of the parameter space. And for the most part, the scale of the problem is what truly matters.
Suppose you have a good computer, and I don’t mean those disappointing laptops or desktop PCs you can buy for semi-reasonable prices on the market which usually come with 16GB of RAM and barely 4 CPU cores, I mean a really good one with say 64GB+ of RAM and say 20 CPU cores. Already, you might think this is enough to solve a certain linear partial differential equation. Say you represent the equation as a linear system of algebraic equations:
L u = b
where u is the solution vector containing all the values at each point of some domain D, b is some source function vector, and L is a linear operator that combines derivatives and other functions into one big matrix. The boundary conditions for u are injected into the equation by subtracting from both sides of the equation, the quantity L’u’ where the prime…
