[mypyc] Use defined __new__ method in tp_new and constructor #19739
Conversation
Co-authored-by: Brian Schubert <brianm.schubert@gmail.com>
| @@ -175,3 +175,6 @@ class _TypedDict(Mapping[str, object]): | |||
|
|
|||
| class TypeAliasType: | |||
| pass | |||
|
|
|||
| def Self(self, parameters): | |||
There was a problem hiding this comment.
We define Self as Self = 0 in other test stubs. Can you try this here?
| @@ -1607,3 +1607,16 @@ def native_class(x): | |||
| L0: | |||
| r0 = CPy_TYPE(x) | |||
| return r0 | |||
|
|
|||
| [case testUnsupportedDunderNew] | |||
There was a problem hiding this comment.
Also add a successful irbuild test that calls super().__new__(cls) in a __new__ method. A simple case is sufficient (i.e. don't return a union type from __new__).
| from m import interpreted_subclass | ||
|
|
||
| @mypyc_attr(allow_interpreted_subclasses=True) | ||
| class Base: |
There was a problem hiding this comment.
Also add a run test that does subclassing but doesn't allow interpreted subclasses.
Add a run test that does subclassing but subclass doesn't override __new__.
mypyc/test-data/run-classes.test
Outdated
| pass | ||
|
|
||
| class Sub(Base): | ||
| def __new__(cls, val: int) -> Base: # type: ignore[misc] |
There was a problem hiding this comment.
Can you avoid using # type: ignore[misc] here? This is something we may disallow in the future, since it can generate unsafe code.
| pass | ||
|
|
||
| def test_inherited_dunder_new() -> None: | ||
| s = Sub(42) |
There was a problem hiding this comment.
Test that type(s) is as expected. Test also constructing an instance of Base, and perform a similar check for it.
mypyc/test-data/run-classes.test
Outdated
| return obj | ||
|
|
||
| def __init__(self, val: int): | ||
| pass |
There was a problem hiding this comment.
Also do something here, to ensure that __init__ is also called? E.g. initialize some other attribute.
mypyc/test-data/run-classes.test
Outdated
| return super().__new__(cls, val + 1) | ||
|
|
||
| def __init__(self, val: int): | ||
| pass |
There was a problem hiding this comment.
Also test that this gets called as expected.
mypyc/test-data/run-classes.test
Outdated
| IntLike = Union[int, Add] | ||
|
|
||
| [file driver.py] | ||
| from native import Add |
There was a problem hiding this comment.
This is only testing behavior in interpreted code. What about also testing that similar things work in compiled code?
| assert t0.val == 0 | ||
| t0 = Test0.__new__(1) | ||
| assert t0.val == 1 | ||
| with assertRaises(TypeError, "__new__() missing required argument 'val'"): |
There was a problem hiding this comment.
Also test a __new__ method that raises an exception in the body (using raise, for example).
| obj = super().__new__(cls) | ||
|
|
||
| with assertRaises(AttributeError, "attribute 'native' of 'Test' undefined"): | ||
| print(obj.native) |
There was a problem hiding this comment.
Also test an attribute of type mypy_extensions.u8 that is undefined (it uses a bitfield to track whether attribute is defined).
Test that an attribute with a default value is initialized correctly (i.e. x = 5 in class body).
Fixes #16012
mypyc ignored custom implementations of
__new__because, even though a C function representing the method was generated, it was called neither in the type constructor nor in the method assigned to thetp_newpointer.Now if there's a
__new__method defined for a type, the corresponding function is called in place of the setup function which is responsible for allocating memory for new objects and initializing their attributes to default values.The setup function is still called when creating instances of the type as calls resolving to
object.__new__()are transformed to call the setup function. This way,__new__can return instances of other types and instances of the type of the class where__new__is defined are setup correctly.There are a couple of limitations:
super().__new__()calls in__new__methods of non-native classes are rejected because it's more difficult to resolve the setup function for non-native classes but this could probably be supported in the future.tp_newmethod of the parent type results in an error because cpython expects the sub type to use a wrapper fortp_newwhich compiled classes don't. Allowing this would require compiled types to be initialized more closely to the way cpython does it which might need a lot of work.__new__is annotated with@classmethod, calling it without the type parameter works in compiled code but raises an error in interpreted. I'm not sure of the reason and it's difficult to make it a compiler error because it's outside of what mypyc sees.