This reverts commit 9865440d.

This is not ready to land yet, seems like the latest update
of the logic reveals a bunch more places I need to fix before it can land.