Python – Copy built-in classes

Copy built-in classes… here is a solution to the problem.

Copy built-in classes

I’m trying to write a function that creates a class from a class without modifying the original class.

Simple solution (based on this answer

).

def class_operator(cls):
    namespace = dict(vars(cls))
    ...  # modifying namespace
    return type(cls.__qualname__, cls.__bases__, namespace)

Works fine, except for type itself:

>>> class_operator(type)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: type __qualname__ must be a str, not getset_descriptor

Test on Python 3.2 – Python 3.6.

(I know that modifying the mutable attribute in the namespace object in the current version changes the original class, but that’s not the case.)

Update

Even if we remove the __qualname__ parameter (if any) from the namespace

def class_operator(cls):
    namespace = dict(vars(cls))
    namespace.pop('__qualname__', None)
    return type(cls.__qualname__, cls.__bases__, namespace)

The resulting object behaves differently than the original type

>>> type_copy = class_operator(type)
>>> type_copy is type
False
>>> type_copy('')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: descriptor '__init__' for 'type' objects doesn't apply to 'type' object
>>> type_copy('empty', (), {})
Traceback (most recent call last):
  File "<input>", line 1, in <module>
TypeError: descriptor '__init__' for 'type' objects doesn't apply to 'type' object

Why?

Can anyone explain what mechanism is inside Python to prevent copying the type class (and many other built-in classes).

Solution

The problem here is that type has an __qualname__ in its __dict__, which is a property (ie descriptor) instead of a string:

>>> type.__qualname__
'type'
>>> vars(type)['__qualname__']
<attribute '__qualname__' of 'type' objects>

And attempting to assign a non-string to a __qualname__ of the class throws an exception:

>>> class C: pass
...
>>> C.__qualname__ = 'Foo'  # works
>>> C.__qualname__ = 3  # doesn't work
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can only assign string to C.__qualname__, not 'int'

That is why it is necessary to remove __qualname__ from the __dict__.

As for why your type_copy is not callable: this is because type.__call__ rejects anything that is not a subclass of type. This is true for all 3 parameters:

>>> type.__call__(type, 'x', (), {})
<class '__main__.x'>
>>> type.__call__(type_copy, 'x', (), {})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: descriptor '__init__' for 'type' objects doesn't apply to 'type' object

As well as the single-argument form, which really only works with type as its first argument:

>>> type.__call__(type, 3)
<class 'int'>
>>> type.__call__(type_copy, 3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: type.__new__() takes exactly 3 arguments (1 given)

This is not easy to circumvent. The fix 3 parameter form is very simple: we make the copy an empty subclass of type.

>>> type_copy = type('type_copy', (type,), {})
>>> type_copy('MyClass', (), {})
<class '__main__. MyClass'>

But the single-argument form of type is more cumbersome because it is only valid if the first argument is type. We can implement a custom __call__ method, but that method must be written in a metaclass, i.e. type(type_copy) and type(type).

>>> class TypeCopyMeta(type):
...     def __call__(self, *args):
...         if len(args) == 1:
...             return type(*args)
...         return super().__call__(*args)
... 
>>> type_copy = TypeCopyMeta('type_copy', (type,), {})
>>> type_copy(3)  # works
<class 'int'>
>>> type_copy('MyClass', (), {})  # also works
<class '__main__. MyClass'>
>>> type(type), type(type_copy)  # but they're not identical
(<class 'type'>, <class '__main__. TypeCopyMeta'>)

Type is so difficult to replicate for two reasons:

  1. It is implemented in C. If you try to copy other built-in types, such as int or str, you will encounter similar problems.
  2. type is an instance of itself:

    >>> type(type)
    <class 'type'>
    

    This is often not possible. It blurs the line between classes and instances. It is a confusing pile of instance and class properties. This is why __qualname__ is a string when accessed as an type.__qualname__ and a descriptor when accessed as vars(type)['__qualname__'].


As you can see, it is impossible to copy the type perfectly. Each implementation has different trade-offs.

The simple solution is to create a subclass of type, which does not support the single-parameter type(some_object) call:

import builtins

def copy_class(cls):
    # if it's a builtin class, copy it by subclassing
    if getattr(builtins, cls.__name__, None) is cls:
        namespace = {}
        bases = (cls,)
    else:
        namespace = dict(vars(cls))
        bases = cls.__bases__

cls_copy = type(cls.__name__, bases, namespace)
    cls_copy.__qualname__ = cls.__qualname__
    return cls_copy

A well-designed solution is to make a custom metaclass:

import builtins

def copy_class(cls):
    if cls is type:
        namespace = {}
        bases = (cls,)

class metaclass(type):
            def __call__(self, *args):
                if len(args) == 1:
                    return type(*args)
                return super().__call__(*args)

metaclass.__name__ = type.__name__
        metaclass.__qualname__ = type.__qualname__
    # if it's a builtin class, copy it by subclassing
    elif getattr(builtins, cls.__name__, None) is cls:
        namespace = {}
        bases = (cls,)
        metaclass = type
    else:
        namespace = dict(vars(cls))
        bases = cls.__bases__
        metaclass = type

cls_copy = metaclass(cls.__name__, bases, namespace)
    cls_copy.__qualname__ = cls.__qualname__
    return cls_copy

Related Problems and Solutions