Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

For PyParallel I just altered the decl for global variables to be TLS static, then tweaked initialization, and voila, thread safe globals.

E.g. https://bitbucket.org/tpn/pyparallel/src/3be2954508f9938b85a...



The problem is that thread local storage is slow. It's an extra layer of indirection. Also, thread safety is just one concern in the GIL debate. One should be able to support many low-overhead independent interpreter instances on a single thread rather than relying on an arcane interpreter context switching scheme as presently exists in Python 2 and 3.


> The problem is that thread local storage is slow.

It's slower than accessing a struct, sure:

    5982:     if (ctx) {
    000000001E1A808B 8B 0D FF 7C 28 00          mov         ecx,dword ptr [_tls_index (1E42FD90h)]  
    000000001E1A8091 BA 80 29 00 00             mov         edx,2980h  
    000000001E1A8096 48 89 83 90 02 00 00       mov         qword ptr [rbx+290h],rax  
    000000001E1A809D 65 48 8B 04 25 58 00 00 00 mov         rax,qword ptr gs:[58h]  
    000000001E1A80A6 48 8B 04 C8                mov         rax,qword ptr [rax+rcx*8]  
    000000001E1A80AA 48 8B 0C 02                mov         rcx,qword ptr [rdx+rax]  
    000000001E1A80AE 48 85 C9                   test        rcx,rcx  
    000000001E1A80B1 74 70                      je          new_context+253h (1E1A8123h)
But think of the big picture: I use TLS everywhere for PyParallel, and PyParallel has awesome performance, so eh, net win.

> One should be able to support many low-overhead independent interpreter instances on a single thread

Why? What problem does that solve?


One use case, totally independent of parallelism is that if someone say embeds Python inside of Postgres or some other large system, I cannot also, in my extension, also embed Python. First one wins and now our systems have to agree. In Lua, you allocate an interpreter context and use that. There can be any number of Lua interpreters embedded in a large system without conflicting.

The use of the globals in the CPython interpreter are a fairly large design mistake that has prevented Python from having as much reach as other systems. The whole embedding vs extending debate is because of those globals and that CPython has been historically difficult to embed properly. Lua on the other hand, is easy to both embed and extend.

I'd love to use `cffi` to embed Python within itself, I cannot do that.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: