Back to articles
What if Python was natively distributable?

What if Python was natively distributable?

via Dev.toconradbzura

The case for distributed execution without orchestration. You have an async function. You want to run it on another machine. How hard could that be? If you’ve gone looking for the answer in the Python ecosystem, you already know: unreasonably so. Not because the problem is complex, but because every framework that offers to solve it insists on solving a dozen other problems you didn’t ask about. Want distributed execution? Great — but first, define your workflow as a DAG. Configure a state backend. Pick a serialization format from a list of four. Set up a message broker. Write a retry policy. Decide on a dead letter queue strategy. Oh, and here’s a decorator, but it only works on top-level functions with JSON-serializable arguments that are registered in a task registry that the worker must import at startup. You wanted to run a function somewhere else. You got an operating system. The quiet assumption There’s a pattern here worth naming. Somewhere along the way, the Python ecosystem d

Continue reading on Dev.to

Opens in a new tab

Read Full Article
2 views

Related Articles