References in the openapi are currently completely hidden from the
model, and just passed through as we walk the tree. The problem is that
they can have a different description and more importantly, different
extensions.

Change them to be first-class citizen, and fully part of the model. It
means that visitors have to implement one more function and decide if
something specific should be done with references. Validation is updated
to just completely ignore them and passthrough (like it was done
before).