Unifying quantum mechanics and general relativity has been a problem for decades. I believe that I have cracked that nut.
Philosophical principle: Laws of physics should not depend on observer’s speed.
Math: Lorentz transform, new way to “add” speeds.
Issues it solved: Maxwell’s equations predict a value for the speed of light that does not depend on your own speed.
Physical observations: The speed of light is indeed independent on observers’ speed (Michelson and Morley’s experiment).
Counter-intuitive aspects: There is no absolute simultaneity and no absolute time. There’s an absolute speed limit for physical objects in the universe.
New requirements: Physicists must now pay attention to the “observer” or “referential”.
Thought experiment: Alice is in a train, while Bob is on the ground watching the train pass him by. What happens if Bob sees a flash hit the train “simultaneously” at both ends? Hint: what happens “at the same time” for Bob is not happening “at the same time” for Alice. That explains why we cannot consider simultaneity as absolute.
Philosophical principle: Laws of physics should not depend on observer’s state of motion, including acceleration.
Math: Non-euclidean geometry, tensor and metrics.
Issues it solved: Discrepancies in the trajectory of Mercury.
Physical observations: Gravitation has an impact on light rays and clocks.
Counter-intuitive aspects: Light has no mass, but is still subject to gravity. The presence of a mass “bends” space-time.
New requirement: Physicists must pay attention to the metric (including curvature) of a given region of space-time.
Typical thought experiment: Alice is in a box on Earth, Bob is in a similar box dragged by a rocket at 1 g. The similarity between their experience explains why we can treat gravitation as a curvature of space-time.
Philosophical principle: Several, “Shut up and calculate” being the top dog today (meaning: if math flies against your intuition, trust the math).
Math: Hilbert spaces, Hamiltonian.
Issues it solved: Black body radiation, structure of matter.
Physical observations: Quantization of light, wave-particle duality, Young’s slits experiment.
Counter-intuitive aspects: Observing something changes it. There are quantities we can’t know at the same time with arbitrary precision, e.g. speed and position of a particle.
New requirement: Physicists must pay attention to what they observe and in which order, as observation may change the outcome of the experiment.
Typical thought experiment: Schrödinger puts his cat in a box where a system built on radioactive decays can kill it at an unknown time in the future. From a quantum mechanical point of view, before you open the box, the cat is in a superposition of two states, alive and dead.
Theory of incomplete measurements:
Philosophical principle: Everything we know about the world, we know from measurements. Laws of physics should be independent from the measurements we chose.
Math: “Meta-math” notation to describe physical experiments independently from the mathematical or symbolical representation of the measurement results. The math of quantum mechanics and general relativity applies only to measurement results, the “meta-math” describes the experiments, including what you measure and what physical section of the universe you use to measure it.
Issues it solved: Unifying quantum mechanics and general relativity. Quantum measurement problem. Why is the wave function complex-valued. Why doesn’t quantum mechanics apply at macroscopic scale (the answer being that it does). Why are there infinities appearing during renormalization, and why is it correct to replace them with observed values?
Physical observations: Room-scale experiments with quantum-like properties. How to transition the definition of the “meter” from a solid rod of matter to a laser beam. Physically different clocks and space measurements diverge at infinity. How can we talk about the probability of a photon being “in the Andromeda galaxy” during a lab experiment? Every measurement of space and time is related to properties of photons. Space-time interpreted as “echolocation with photons”.
Counter-intuitive aspects: Quantum mechanics is the necessary form of physics when we deal with probabilistic knowledge of the world. In most cases, our knowledge of the world is probabilistic. All measurements are not equivalent, and a “better” measurement (i.e. higher resolution) is not universally better (i.e. it may not correctly extend a lower-resolution but wider scale measurement). Space-time (and all measurements) are quantized. There is no pre-existing “continuum”, the continuum is a mathematical simplification we introduce to unify physically different measurements of the same thing (e.g. distance measurements by our eye and by pocket rulers).
New requirement: Physicists must specify which measurement they use and how two measurements of the “same thing” (e.g. mass) are calibrated to match one another.
Typical thought experiment: Measure the earth surface with the reference palladium rod, and then with a laser. Both methods were at some point used to define the “meter” (i.e. distance). Why don’t they bend the same way under gravitational influence? In that case, the Einstein tensors and metrics would be different based on which measurement “technology” you used.
So how does the unification happen?
To illustrate how the unification happens without too much math, imagine a biologist trying to describe the movement of ants on the floor.
The “quantum mechanical” way to do it to compute the probability of having an ant at each location. The further away from the ants’ nest, the lower the probability. Also, the probability to find an ant somewhere is related to the probability of finding it someplace near a short time before. When you try to setup the “boundary conditions” for these probabilities, you will say something like: the ant has to be somewhere, so the probability summed over all of space is one; and the probability becomes vanishingly small “at infinity”.
The general-relativistic way to do it will consider the trajectories of the ants on the 2D surface. But to be very precise, it will need to take into account the fact that ants are on a large-scale sphere, and deduce that the 2D surface they walk on is not flat (euclidean) but curved. For example, if an ant travelled along the edges of a 1000km square (from its point of view), it would not return exactly where it left off, therefore proving that the 2D surface is not flat.
At a relatively small scale, the two approaches can be made to coincide almost exactly. But they diverge in their interpretation of “at infinity”. Actually, assuming observed ants stay within a radius R of the nest, there are an infinite number of coordinate systems that are equal on that radius R, but diverge beyond R. Of course, the probabilities you compute depend on the coordinate system.
In particular, if you take a “curved” coordinate systems that loops around the earth to match the “general relativistic” view of the world, the physically observed probability does not match the original idea we have that probability becomes vanishingly small at infinity and that the sum is one. In that physical coordinates system, the probability to see ants is periodically non-zero (every earth circumference, you see the same ant “again”). So your integral and probability computation is no longer valid. It shows false infinities that are not observed in the physical world. You need to “renormalize” it.
In the theory of incomplete measurements, you focus on probabilities like in quantum mechanics, but only on the possible measurement results of your specific physical measurement system. If your measurement system follows the curvature of earth (e.g. you use solid rods of matter), then the probabilities will be formally different from a measurement system that does not follow it (e.g. you use laser beams). Key topological or metric properties therefore depend on the measurement apparatus being chosen. There is no “x” in the equations that assumes some underlying space-time with specific topology or metric. Instead, there is a “x as measured by this apparatus”, with the topology and metric that derives from the given apparatus.
Furthermore, all the probabilities will be computed using finite sums, because all known measurement instruments give only finite measurement results. There may be a “measurement not valid” probability bin. But if you are measuring the position of a photon in a lab, there cannot be a “photon was found in the Andromeda galaxy” probability bin (unlike in quantum mechanics), because your measurement apparatus simply cannot detect your photon in the Andromeda galaxy. Such a probability is non-sensical from a physical point of view, so we build the math to exclude it.
So in the theory of incomplete measurements, you only have finite sums that cannot diverge, and renormalisation is the mathematical equivalent of calibrating physically different measurement instruments to match one another.
The analogy is not perfect, but in my opinion, it explains relatively well what happens with as little math as possible.