Interesting problem

I am trying to flesh out the thread locking for our embedded python interpreter. A user can write scripts for our application, and if a script defines a callback it can respond to events from the application (like a midi note or musical clock tick).

As it stands, we are not using process migration to allow for true multiprocessing, but instead are just hamming a single lock from multiple threads to protect the CPython library. This is good enough for now – it works quite well at relatively low latencies.

I compiled python without thread management to avoid an incredible number of ‘NULL tstate’ errors caused by unpredictable thread management by sequencing host applications. The app is solid, but now I want to integrate a simple stack frame mechanism to allow scripts to behave a little better when calling other scripts. Short intro, but the details don’t matter anyway…

Here’s the deal

I have one audio engine per thread, and several scripts per engine. The currently executing script is stored as a pointer on the engine object using my function setActiveScript(), and when a new script is called it is swapped with the current one, much like PyThreadState_Swap(). This in effect creates a stack of scripts in the engine, where when execution returns to the original script NULL is passed to setActiveScript().

This works great, but when there is more than one engine (and so more than one thread), I need to acquire a critical section lock and swap the engine for another one, while maintaining the current script on that engine. The script pointer is stored in the engine, but where to enter and exit the critical section is tough. Hairy.

The function looks like this:

Script *ScriptingApplication::instance()->setActiveScript(Script *)

I haven’t written much complex code in the last while, so lining out the algorithm and thread locks is kind of clunky at best. it doesn’t help that Ableton Live (a host that loads our audio plugin) suddenly doesn’t want to be debugged.

*crash*. uggh.