Back to articles
1m Tokens (& WebSocket)

1m Tokens (& WebSocket)

via Dev.to PythonJoseph Boone

Greetings readers, I made a threading engine with many optimizations (including ML) and WebSocket task controls per operation. Even when computing a slow moving series like Leibniz PI series at 1 million token executions the tasks all resolved as expected in ~200 seconds. # ── LAYER 0: TERM TOKENS ────────────────────────────────────────────────────── @task_token_guard ( operation_type = ' pi_term ' , tags = { ' weight ' : ' light ' }) def compute_pi_term ( n : int ) -> str : """ Compute a single Leibniz term: (-1)^n / (2n + 1) Returns as string to preserve Decimal precision across token boundary. Light weight — 1,000,000 of these fire simultaneously. """ getcontext (). prec = DECIMAL_PRECISION sign = Decimal ( - 1 ) ** n term = sign / Decimal ( 2 * n + 1 ) return str ( term ) # ── LAYER 1: CHUNK TOKENS ───────────────────────────────────────────────────── @task_token_guard ( operation_type = ' pi_chunk ' , tags = { ' weight ' : ' light ' }) def sum_chunk ( term_strings : List [ str ])

Continue reading on Dev.to Python

Opens in a new tab

Read Full Article
2 views

Related Articles