Consider this problem:
Your C++ project includes libpython and special multiprocessing constraints require you to move the CPython interpreter code and GIL out of the main process. No, you can’t use multiprocessing or any other python-based code because it requires the GIL. The GIL must move. The GIL will move.
You can use your nifty CS degree to easily abstract your use of the python interpreter into an abstract interface. Communication with the interface is easily serializable. You’re well on your way.
But wait, there’s one problem. Clean, tight process management is something you’ve never worked with before – not to mention on Windows and Mac. Your requirements are simple, but the challenge is hard.
“All I want to do is start a process, get a read and a write file descriptor, and kill the process.” Sorry, but it’s not that simple. Unix has sockets, Windows does not. They both use radically different calls to manage processes. And what about shared memory? Sheesh…scary.
You won’t be firing off new processes very often, but you can’t afford hung processes. You need 100% reliability. You can’t afford much time to waste time debugging runaway processes. You just need the basic process management code to work – and all in C++.
How simple can this be? Surely there is a library for this? is it just a little code to copy and maintain myself? Seems like it should be.
If we are going to leave the GIL as is, shouldn’t it’ be a no-brainer to attack a problem like this? Is there a C++ version of multiprocessing, or at least something close? If we are going to accept process migration as a viable solution to true python multiprocessing, shouldn’t we remove problems like this from the equation?
Can we simply do this with better tools? Should we just do this with other tools? I think so.
The CPython interpreter is fast. It’s like, really super fast, and we use it in a real-time audio application without so much as a blip on the profiler. It is such a shame that such a fast, clean language can’t be used by our several threads without overcoming this crazy process migration hurdle. I’d love to move it to a new process, but first I have to find out how!
how about spawn a python interpreter in a thread within your main C++ process (create the thread before your create the Python VM). This interpreter is confined to this thread and it’s GIL doesn’t interfere with your C++ code. Create pipes for communication with between the C++ thread and the VM. I.e. use a thread but treat it like a process. You can then spawn extra processes from this Python thread using multiprocessing to do concurrent python work.
What do you mean exactly by “move the CPython interpreter code and GIL out of the main process”?
Do you mean you want to migrate the interpreter from one process to another? If so, why do you want to do that? And why do you expect it to be even possible? 🙂
I am a little confused by exactly what you want to do as well.
I think I had a similar problem on the last python & C++ project I worked on. We had a multi-threaded C++ system integrating with python. Since the python VM has the GIL would could not have our soft-relatime code calling into python and blocking. I used my yet unearned CS degree to decouple the two halves of the system with an event system. & queue Python can call into C++ all it wants, but C++ reports information back to the python side in events which get queued up for consumption when the python thread gets around to it.
If you want cross platform process management in C++, I would check out the Boost.Process candidate library (not yet in boost).
オナニー
逆援助
SEX
フェラチオ
ソープ
逆援助
出張ホスト
手コキ
おっぱい
フェラチオ
中出し
セックス
デリヘル
包茎
逆援
性欲