Python3.12 介绍

November 21, 2023

wKevin

一颗向上的水滴

摘要：

python3.12 与 2023.10 发布正式版：3.12.0，开始进入生产阶段的生命周期。

本文从 jupyter 导出，每段代码后面跟随的是代码的执行结果。

环境搭建

安装 python（下面方法 2 选 1）

pyenv & venv

$ pyenv install 3.12.0
$ pyenv versions
$ pyenv shell 3.12.0
$ python -V
$ python -m venv venv312
$ source venv312/bin/activate

docker

# 替换其中的 `<my_code>` 为自己代码所在位置
$ docker run -it -d --name py312 -p 8888:8888 -v <my_code>:/src python:3.12
$ docker exec -it py312 bash

安装 jupyter（下面方法 2 选 1） —— 可选
1. jupyter lab/notebook on browser
  - pip install jupyter
  - pip install nb_mypy black isort jupyterlab-code-formatter jupyter-black —— 可选
  - jupyter lab --allow-root --no-browser --ip=0.0.0.0 or jupyter notebook --allow-root --no-browser --ip=0.0.0.0
  - 使用带有 token 的链接打开浏览器
  - 配置：Settings -- Settings Editor
    - jupyterlab-code-formatter -- （拉到最下面）-- Auto format config -- 勾选
2. jupyter notebook on VSCode
  - 安装 VSCode Jupyter 扩展
  - F1 - select interpreter - 选择 python venv —— 这样在 jupyter 中 kernel 才能被看到
  - 打开 ipynb 文件，右上角选择解释器

PEP 684: 解析器级 GIL

从 1997 年的 python v1.5 开始，可以在一个进程中运行多个解释器，但这些解释器共用一个 GIL（全局进程锁），所以虽然多线程，但其实是单线程。

26 年之后，python3.12 终于从进程级 GIL 改为解释器级 GIL：PEP 684 – A Per-Interpreter GIL

每个解释器可以创建自己独立的 GIL：

一个解释器上的多线程仍然共用一个 GIL
多个解释器上的多线程使用各自的 GIL

API 预计将在 3.13 中添加: PEP 554 – Multiple Interpreters in the Stdlib

新增 interpreters 模块
新增 class conCurent.futures.InterpreterPoolExecutor

Demo:

import interpreters

interp = interpreters.create()

def run():
    interp.exec('print("during")')

t = threading.Thread(target=run)

t.start()
t.join()

PEP 701：f-字符串语法改进

python3.6 首次引入了 f 字符串语法：PEP 498 – Literal String Interpolation
python3.7 中做了一些优化：PEP 536 – Final Grammar for Literal String Interpolation
python3.12 中做个更多的优化，主要是放弃了一些写法限制：PEP 701 – Syntactic formalization of f-strings

允许在表达式部分中使用引号字符（引号重用）

facebook = {'Tom': 60, 'Jerry': 90, "Bloto": 30}

print(f'Tom score: { facebook['Tom'] }')
print(f"Jerry score: { facebook["Jerry"] }")
print(f"Bloto score: { facebook['Bloto'] }")

# print(f'Tom score: { facebook[\'Tom\'] }') # python3.12 之前 禁止 f 中转义字符

Tom score: 60
Jerry score: 90
Bloto score: 30

songs = ["Take me back to Eden", "Alkaline", "Ascensionism"]
print(f"This is the playlist: {", ".join(songs)}")

# py3.12 之前可以这样实现：
print(f"This is the playlist: {', '.join(songs)}") # 不同的引号
print(f"""This is the playlist: {', '.join(songs)}""") # 不同优先级的引号做嵌套

This is the playlist: Take me back to Eden, Alkaline, Ascensionism
This is the playlist: Take me back to Eden, Alkaline, Ascensionism
This is the playlist: Take me back to Eden, Alkaline, Ascensionism

允许写 \ 字符

Unicode 字符：\N{...}
- 这里有各种语言对应的编码
转义字符：\r、\n、\b、\t、\n

print(f"This is the playlist: {" \N{BLACK HEART SUIT} ".join(songs)}")
print(f"\N{snowman} \N{Grinning Face} \N{Face With Tears Of Joy} \U0001F602 ")

This is the playlist: Take me back to Eden ♥ Alkaline ♥ Ascensionism
☃ 😀 😂 😂

print(f"This is the playlist: {"\n".join(songs)}")

This is the playlist: Take me back to Eden
Alkaline
Ascensionism

from time import sleep

starts = ['***', '++', '-']

print(f"{"\r".join(starts)}")

-+*

嵌套方式更加直观，嵌套层级扩展到无限

# python3.6 开始，可以这样写嵌套，4层嵌套也是最多的了
print(f"""{f'''{f'{f"{1+1}"}'}'''}""")

2

# python3.12 开始，可以这样写任意多层的嵌套了
print(f"{f"{f"{f"{f"{f"{1+1}"}"}"}"}"}")

print(f"{f"{f"{f"{f"{f"{1+1}"+"a"}"+"b"}"+"c"}"+"d"}"+"e"}")

2
2abcde

可以定义跨越多行的 f-字符串并添加内联注释

print(f"Flake8 error code: {", ".join([
    'F401', # module imported but unused
    'F402', # import module from line N shadowed by loop variable
    'F403', # ‘from module import *’ used; unable to detect undefined names
    'F404', # future import(s) name after other statements
])}")

Flake8 error code: F401, F402, F403, F404

可以在表达式组件中包含任何有效的 Python 表达式

f"{print("abc")}"

abc

一些技巧

# 从 python3.6 开始，就支持 :

f"some words {1+2:.3f}"

'some words 3.000'

# 从 python3.8 开始，f字符串可以通过使用 = 运算符来调试表达式

f"{1+1=}"

'1+1=2'

PEP 669: CPython 监控器

PEP 669 – Low Impact Monitoring for CPython

sys.monitoring 是 sys 模块中的命名空间，而不是独立的模块，因此:

不能：import sys.monitoring 或 from sys import monitoring
而是：import sys 后使用 sys.monitoring

monitoring 有 3 个组件：

Tool identifiers
Events
Callbacks

4 个 Tool, 3 个函数

import sys

# 4 个 tool
# sys.monitoring.DEBUGGER_ID = 0
# sys.monitoring.COVERAGE_ID = 1
# sys.monitoring.PROFILER_ID = 2
# sys.monitoring.OPTIMIZER_ID = 5

# 3 个函数
sys.monitoring.free_tool_id(sys.monitoring.DEBUGGER_ID)
sys.monitoring.use_tool_id(sys.monitoring.DEBUGGER_ID, "foobar")
sys.monitoring.get_tool(sys.monitoring.DEBUGGER_ID)

10: error: Module has no attribute "monitoring" [attr-defined]
ERROR:nb-mypy:10: error: Module has no attribute "monitoring" [attr-defined]
11: error: Module has no attribute "monitoring" [attr-defined]
ERROR:nb-mypy:11: error: Module has no attribute "monitoring" [attr-defined]
12: error: Module has no attribute "monitoring" [attr-defined]
ERROR:nb-mypy:12: error: Module has no attribute "monitoring" [attr-defined] 'foobar'

19 个 Event，6 个函数

Event 分为 3 组：

Local events：

sys.monitoring.events.PY_START —— Python 函数的开始
sys.monitoring.events.PY_RESUME
sys.monitoring.events.PY_RETURN —— 从 Python 函数返回
sys.monitoring.events.PY_YIELD
sys.monitoring.events.CALL
sys.monitoring.events.LINE —— 即将执行的指令的行号与前一条指令不同
sys.monitoring.events.INSTRUCTION
sys.monitoring.events.JUMP
sys.monitoring.events.BRANCH
sys.monitoring.events.STOP_ITERATION

Ancillary events

sys.monitoring.events.C_RAISE —— 非 python 函数抛异常
sys.monitoring.events.C_RETURN

Other events

sys.monitoring.events.PY_THROW
sys.monitoring.events.PY_UNWIND —— 在异常展开期间退出 Python 函数
sys.monitoring.events.RAISE —— python 函数抛异常
sys.monitoring.events.RERAISE
sys.monitoring.events.EXCEPTION_HANDLED —— 捕获异常
sys.monitoring.events.NO_EVENTS
sys.monitoring.DISABLE

6 个函数：

sys.monitoring.set_events(tool_id: int, event_set: int, /) → None
- 可以或操作：RAISE | PY_START
sys.monitoring.get_events(tool_id: int, /) → int
sys.monitoring.get_local_events(tool_id: int, code: CodeType, /) → int
sys.monitoring.set_local_events(tool_id: int, code: CodeType, event_set: int, /) → None
sys.monitoring.restart_events() → None
sys.monitoring.register_callback(tool_id: int, event: int, func: Callable | None, /) → Callable | None —— 注册回调函数，不同的 Event 回调函数签名不同

import sys
from types import CodeType

M = sys.monitoring
TD = M.DEBUGGER_ID # 0
TC = M.COVERAGE_ID # 1
TP = M.PROFILER_ID # 2
TO = M.OPTIMIZER_ID # 5
E = sys.monitoring.events

class CounterWithDisable:
    def __init__(self):
        self.disable = False
        self.count = 0

    def __call__(self, *args):
        if self.count  == 100:
            print(args)
        self.count += 1
        if self.disable:
            return sys.monitoring.DISABLE


def foobar():
    pass

tool = TC

counter = CounterWithDisable()
print(counter.count)
sys.monitoring.register_callback(tool, E.LINE, counter) # 注册回调

if sys.monitoring.get_tool(tool) is not None: # Tool 不能重复启用
    sys.monitoring.free_tool_id(tool)
sys.monitoring.use_tool_id(tool, "ToolP") # 启用 Tool

sys.monitoring.set_events(tool, E.LINE | E.PY_RETURN)
print(f"events: {sys.monitoring.get_events(tool)}")
print(counter.count)
foobar()
print(counter.count)

0
/home/me/.pyenv/versions/3.12.0/lib/python3.12/contextlib.py
events: 36
412
594

PEP 695: 泛型新语法

PEP 695 – Type Parameter Syntax

# 在此PEP之前定义泛型类如下所示

from typing import Generic, TypeAlias, TypeVar

_T_co = TypeVar("_T_co", covariant=True, bound=str)

class ClassA(Generic[_T_co]):
    def method1(self) -> _T_co:
        ...

_T = TypeVar("_T")

def func(a: _T, b: _T) -> _T:
    ...

ListOrSet: TypeAlias = list[_T] | set[_T]

10: error: Incompatible return value type (got "str", expected "_T_co") [return-value]
ERROR:nb-mypy:10: error: Incompatible return value type (got "str", expected "_T_co") [return-value]
16: error: Missing return statement [empty-body]
ERROR:nb-mypy:16: error: Missing return statement [empty-body]

# 使用新的语法，它看起来如下所示

class ClassA[T: str]:
    def method1(self) -> T:
        ...

def func[T](a: T, b: T) -> T:
    ...

type ListOrSet[T] = list[T] | set[T]

4: error: PEP 695 generics are not yet supported [valid-type]
ERROR:nb-mypy:4: error: PEP 695 generics are not yet supported [valid-type]
5: error: Missing return statement [empty-body]
ERROR:nb-mypy:5: error: Missing return statement [empty-body]
5: error: A function returning TypeVar should receive at least one argument containing the same TypeVar [type-var]
ERROR:nb-mypy:5: error: A function returning TypeVar should receive at least one argument containing the same TypeVar [type-var]
9: error: Missing return statement [empty-body]
ERROR:nb-mypy:9: error: Missing return statement [empty-body]
9: error: PEP 695 generics are not yet supported [valid-type]
ERROR:nb-mypy:9: error: PEP 695 generics are not yet supported [valid-type]
13: error: PEP 695 type aliases are not yet supported [valid-type]
ERROR:nb-mypy:13: error: PEP 695 type aliases are not yet supported [valid-type]

typing 模块修改

PEP 698: 新增 `@override()`

# 向类型检查器指示该方法旨在重写超类中的方法。
# 这允许类型检查器在打算重写基类中的某个方法实际上没有重写的情况下捕获错误。
from typing import override


class Base:
    def get_color(self) -> str:
        return "blue"


class GoodChild(Base):
    @override  # ok: overrides Base.get_color
    def get_color(self) -> str:
        return "yellow"


class BadChild(Base):
    @override  # type checker error: does not override Base.get_color
    def get_colour(self) -> str:
        return "red"

19: error: Method "get_colour" is marked as an override, but no base method was found with this name [misc]
ERROR:nb-mypy:19: error: Method "get_colour" is marked as an override, but no base method was found with this name [misc]

PEP 692: 优化 `class TypedDict(dict)`

Python3.8 引入 TypedDict。

创建用户的字典时，使用 TypedDict 做基类，linter 可完成正确的类型检查。

Python3.12 优化了 **kwargs 的类型标注方式 的标注方式：PEP 692 – Using TypedDict for more precise **kwargs typing

from typing import TypedDict


class Point2D(TypedDict):
    x: int
    y: int
    label: str


a: Point2D = {"x": 1, "y": 2, "label": "good"}  # OK
b: Point2D = {"z": 3, "label": "bad"}  # 类型检查失败

print(Point2D(x=1, y=2, label="first") == dict(x=1, y=2, label="first"))  # 运行时即dict

11: error: Missing keys ("x", "y") for TypedDict "Point2D" [typeddict-item]
ERROR:nb-mypy:11: error: Missing keys ("x", "y") for TypedDict "Point2D" [typeddict-item]
11: error: Extra key "z" for TypedDict "Point2D" [typeddict-unknown-key]
ERROR:nb-mypy:11: error: Extra key "z" for TypedDict "Point2D" [typeddict-unknown-key]

True

from typing import Unpack


# python3.12 新增了 **kwargs 的类型标注方式，类型识别更精准
def foobar(**kwargs: Unpack[Point2D]):
    ...

移除

distuils 包

py3.10 开始启用，py3.12 移除。

pyTP 中尚在使用，需要尽快考虑替换。

# setup.py
from distutils.core import Command

推荐的 pip 安装器用 setuptools 运行所有的 setup.py 脚本

imp 模块

使用 importlib 模块替代其功能。

环境搭建​

PEP 684: 解析器级 GIL​

PEP 701：f-字符串语法改进​

PEP 669: CPython 监控器​

PEP 695: 泛型新语法​

typing 模块修改​

PEP 698: 新增 @override()​

PEP 692: 优化 class TypedDict(dict)​

移除​

distuils 包​

imp 模块​