Can you use AMP tags on non-amp pages?

More specifically, can you use AMP tags and code on pages that do not pass AMP validation?

Use case: I want to make my website entirely using AMP, and to do that, I will need to use things like on='tap:' for my dropdown menu and <amp-analytics> for tracking, but there are still components of my site that are not ready for AMP.

Can my site be live, relying on these AMP technologies, even though the site is not fully AMP compliant quite yet?

EDIT: This question is unique in that I intend to become AMP compliant, but I'm waiting for a few changes to be made before I do, until then I want my site to be live and not indexed as AMP by Google, but still made available in Google search. The best answer below does address this.

Solution

Yes. That's possible. However, there are a few caveats:

make sure to not set the amp attribute on the html element. This avoids GoogleBot treating your page as an AMP page.
AMP validation will no longer work which makes it harder to detect errors.
AMP controls the site layout and needs to know the height of each element on the page. If using custom JS on your page, each non-AMP element should have a fixed height. This avoids breaking AMP's layout assumptions.

To make this use case easier in the future, the AMP team is working on Bento AMP, the ability to use AMP components in an “un-managed” way, without loading AMP’s main JS file (v0.js), and coexisting with other web components and frameworks on the same page.