establish mypy / typing approach for v2.0

large patch to get ORM / typing efforts started.
this is to support adding new test cases to mypy,
support dropping sqlalchemy2-stubs entirely from the
test suite, validate major ORM typing reorganization
to eliminate the need for the mypy plugin.

* New declarative approach which uses annotation
  introspection, fixes: #7535
* Mapped[] is now at the base of all ORM constructs
  that find themselves in classes, to support direct
  typing without plugins
* Mypy plugin updated for new typing structures
* Mypy test suite broken out into "plugin" tests vs.
  "plain" tests, and enhanced to better support test
  structures where we assert that various objects are
  introspected by the type checker as we expect.
  as we go forward with typing, we will
  add new use cases to "plain" where we can assert that
  types are introspected as we expect.
* For typing support, users will be much more exposed to the
  class names of things.  Add these all to "sqlalchemy" import
  space.
* Column(ForeignKey()) no longer needs to be `@declared_attr`
  if the FK refers to a remote table
* composite() attributes mapped to a dataclass no longer
  need to implement a `__composite_values__()` method
* with_variant() accepts multiple dialect names

Change-Id: I22797c0be73a8fbbd2d6f5e0c0b7258b17fe145d
Fixes: #7535
Fixes: #7551
References: #6810

Author: zzzeek

doc/build/changelog/migration_20.rst

@@ -182,7 +182,7 @@ and pylance.  Given a program as below::
     from sqlalchemy.dialects.mysql import VARCHAR
 
 
-    type_ = String(255).with_variant(VARCHAR(255, charset='utf8mb4'), "mysql")
+    type_ = String(255).with_variant(VARCHAR(255, charset='utf8mb4'), "mysql", "mariadb")
 
     if typing.TYPE_CHECKING:
         reveal_type(type_)
@@ -191,6 +191,9 @@ A type checker like pyright will now report the type as::
 
     info: Type of "type_" is "String"
 
+In addition, as illustrated above, multiple dialect names may be passed for
+single type, in particular this is helpful for the pair of ``"mysql"`` and
+``"mariadb"`` dialects which are considered separately as of SQLAlchemy 1.4.
 
 :ticket:`6980`
 

doc/build/changelog/unreleased_20/6980.rst

@@ -10,6 +10,10 @@
     behaviors, maintaining the original type allows for clearer type checking
     and debugging.
 
+    :meth:`_sqltypes.TypeEngine.with_variant` also accepts multiple dialect
+    names per call as well, in particular this is helpful for related
+    backend names such as ``"mysql", "mariadb"``.
+
     .. seealso::
 
         :ref:`change_6980`

doc/build/changelog/unreleased_20/composite_dataclass.rst

@@ -0,0 +1,9 @@
+.. change::
+    :tags: feature, orm
+
+    The :func:`_orm.composite` mapping construct now supports automatic
+    resolution of values when used with a Python ``dataclass``; the
+    ``__composite_values__()`` method no longer needs to be implemented as this
+    method is derived from inspection of the dataclass.
+
+    See the new documentation at :ref:`mapper_composite` for examples.

doc/build/changelog/unreleased_20/decl_fks.rst

@@ -0,0 +1,9 @@
+.. change::
+    :tags: feature, orm
+
+    Declarative mixins which use :class:`_schema.Column` objects that contain
+    :class:`_schema.ForeignKey` references no longer need to use
+    :func:`_orm.declared_attr` to achieve this mapping; the
+    :class:`_schema.ForeignKey` object is copied along with the
+    :class:`_schema.Column` itself when the column is applied to the declared
+    mapping.

doc/build/changelog/unreleased_20/prop_name.rst

@@ -0,0 +1,19 @@
+.. change::
+    :tags: change, orm
+
+    To better accommodate explicit typing, the names of some ORM constructs
+    that are typically constructed internally, but nonetheless are sometimes
+    visible in messaging as well as typing, have been changed to more succinct
+    names which also match the name of their constructing function (with
+    different casing), in all cases maintaining aliases to the old names for
+    the forseeable future:
+
+    * :class:`_orm.RelationshipProperty` becomes an alias for the primary name
+      :class:`_orm.Relationship`, which is constructed as always from the
+      :func:`_orm.relationship` function
+    * :class:`_orm.SynonymProperty` becomes an alias for the primary name
+      :class:`_orm.Synonym`, constructed as always from the
+      :func:`_orm.synonym` function
+    * :class:`_orm.CompositeProperty` becomes an alias for the primary name
+      :class:`_orm.Composite`, constructed as always from the
+      :func:`_orm.composite` function

doc/build/core/selectable.rst

@@ -156,21 +156,13 @@ Label Style Constants
 Constants used with the :meth:`_sql.GenerativeSelect.set_label_style`
 method.
 
-.. autodata:: LABEL_STYLE_DISAMBIGUATE_ONLY
+.. autoclass:: SelectLabelStyle
+    :members:
 
-.. autodata:: LABEL_STYLE_NONE
 
-.. autodata:: LABEL_STYLE_TABLENAME_PLUS_COL
+.. seealso::
 
-.. data:: LABEL_STYLE_DEFAULT
+    :meth:`_sql.Select.set_label_style`
 
-  The default label style, refers to :data:`_sql.LABEL_STYLE_DISAMBIGUATE_ONLY`.
-
-  .. versionadded:: 1.4
-
-  .. seealso::
-
-      :meth:`_sql.Select.set_label_style`
-
-      :meth:`_sql.Select.get_label_style`
+    :meth:`_sql.Select.get_label_style`
 

doc/build/orm/composites.rst

@@ -5,6 +5,11 @@
 Composite Column Types
 ======================
 
+.. note::
+
+    This documentation is not yet updated to illustrate the new
+    typing-annotation syntax or direct support for dataclasses.
+
 Sets of columns can be associated with a single user-defined datatype. The ORM
 provides a single attribute which represents the group of columns using the
 class you provide.

doc/build/orm/declarative_mixins.rst

@@ -125,47 +125,61 @@ for each separate destination class.  To accomplish this, the declarative
 extension creates a **copy** of each :class:`_schema.Column` object encountered on
 a class that is detected as a mixin.
 
-This copy mechanism is limited to simple columns that have no foreign
-keys, as a :class:`_schema.ForeignKey` itself contains references to columns
-which can't be properly recreated at this level.  For columns that
-have foreign keys, as well as for the variety of mapper-level constructs
-that require destination-explicit context, the
-:class:`_orm.declared_attr` decorator is provided so that
-patterns common to many classes can be defined as callables::
+This copy mechanism is limited to :class:`_schema.Column` and
+:class:`_orm.MappedColumn` constructs. For :class:`_schema.Column` and
+:class:`_orm.MappedColumn` constructs that contain references to
+:class:`_schema.ForeignKey` constructs, the copy mechanism is limited to
+foreign key references to remote tables only.
+
+.. versionchanged:: 2.0 The declarative API can now accommodate
+   :class:`_schema.Column` objects which refer to :class:`_schema.ForeignKey`
+   constraints to remote tables without the need to use the
+   :class:`_orm.declared_attr` function decorator.
+
+For the variety of mapper-level constructs that require destination-explicit
+context, including self-referential foreign keys and constructs like
+:func:`_orm.deferred`, :func:`_orm.relationship`, etc, the
+:class:`_orm.declared_attr` decorator is provided so that patterns common to
+many classes can be defined as callables::
 
     from sqlalchemy.orm import declared_attr
 
     @declarative_mixin
-    class ReferenceAddressMixin:
+    class HasRelatedDataMixin:
         @declared_attr
-        def address_id(cls):
-            return Column(Integer, ForeignKey('address.id'))
+        def related_data(cls):
+            return deferred(Column(Text())
 
-    class User(ReferenceAddressMixin, Base):
+    class User(HasRelatedDataMixin, Base):
         __tablename__ = 'user'
         id = Column(Integer, primary_key=True)
 
-Where above, the ``address_id`` class-level callable is executed at the
+Where above, the ``related_data`` class-level callable is executed at the
 point at which the ``User`` class is constructed, and the declarative
-extension can use the resulting :class:`_schema.Column` object as returned by
+extension can use the resulting :func`_orm.deferred` object as returned by
 the method without the need to copy it.
 
-Columns generated by :class:`_orm.declared_attr` can also be
-referenced by ``__mapper_args__`` to a limited degree, currently
-by ``polymorphic_on`` and ``version_id_col``; the declarative extension
-will resolve them at class construction time::
+For a self-referential foreign key on a mixin, the referenced
+:class:`_schema.Column` object may be referenced in terms of the class directly
+within the :class:`_orm.declared_attr`::
 
-    @declarative_mixin
-    class MyMixin:
-        @declared_attr
-        def type_(cls):
-            return Column(String(50))
+        class SelfReferentialMixin:
+            id = Column(Integer, primary_key=True)
 
-        __mapper_args__= {'polymorphic_on':type_}
+            @declared_attr
+            def parent_id(cls):
+                return Column(Integer, ForeignKey(cls.id))
+
+        class A(SelfReferentialMixin, Base):
+            __tablename__ = 'a'
 
-    class MyModel(MyMixin, Base):
-        __tablename__='test'
-        id =  Column(Integer, primary_key=True)
+
+        class B(SelfReferentialMixin, Base):
+            __tablename__ = 'b'
+
+Above, both classes ``A`` and ``B`` will contain columns ``id`` and
+``parent_id``, where ``parent_id`` refers to the ``id`` column local to the
+corresponding table ('a' or 'b').
 
 .. _orm_declarative_mixins_relationships:
 
@@ -182,9 +196,7 @@ reference a common target class via many-to-one::
 
     @declarative_mixin
     class RefTargetMixin:
-        @declared_attr
-        def target_id(cls):
-            return Column('target_id', ForeignKey('target.id'))
+        target_id = Column('target_id', ForeignKey('target.id'))
 
         @declared_attr
         def target(cls):

doc/build/orm/internals.rst

@@ -32,9 +32,10 @@ sections, are listed here.
 
             :ref:`maptojoin` - usage example
 
-.. autoclass:: CompositeProperty
+.. autoclass:: Composite
     :members:
 
+.. autodata:: CompositeProperty
 
 .. autoclass:: AttributeEvent
     :members:
@@ -62,6 +63,8 @@ sections, are listed here.
 
 .. autoclass:: Mapped
 
+.. autoclass:: MappedColumn
+
 .. autoclass:: MapperProperty
     :members:
 
@@ -98,14 +101,18 @@ sections, are listed here.
     :members:
     :inherited-members:
 
-.. autoclass:: RelationshipProperty
+.. autoclass:: Relationship
     :members:
     :inherited-members:
 
-.. autoclass:: SynonymProperty
+.. autodata:: RelationshipProperty
+
+.. autoclass:: Synonym
     :members:
     :inherited-members:
 
+.. autodata:: SynonymProperty
+
 .. autoclass:: QueryContext
     :members:
 

doc/build/orm/loading_relationships.rst

@@ -1261,8 +1261,6 @@ Relationship Loader API
 
 .. autofunction:: defaultload
 
-.. autofunction:: eagerload
-
 .. autofunction:: immediateload
 
 .. autofunction:: joinedload

doc/build/orm/relationship_api.rst

@@ -7,8 +7,6 @@ Relationships API
 
 .. autofunction:: backref
 
-.. autofunction:: relation
-
 .. autofunction:: dynamic_loader
 
 .. autofunction:: foreign