The strategy the allocator uses is to reserve a large contiguous block of address space(not memory) on start up and then allocate actual memory from that block.
On 32bit this is around 512MB, on 64bit it's something like 16TB. This makes allocating and garbage collection far simpler and it really shouldn't be a big ask for an operating system to give a process a contiguous block of address space and not interfere with it. But just like the pinning issue the smaller address space on 32bit means that things that expect to be at a certain address have a higher likelihood of getting in the way of the contiguous block.
The fix for both this issue and the pinning issue is big changes to the garbage collector. Since the language and standard library have stabilised with Go1, changes to the runtime are going to be the focus of development so this issue will probably get a fix.
This is highly dependent on the video card and other PCI devices you are using, as they will often reserve huge chunks of VA space to map to device memory (and furthermore, require it to be in low addresses i.e. user space), making a contiguous reservation difficult.
Getting 512MB of contiguous virtual address space does not require 512MB of contiguous free physical memory; contiguous virtual addresses can map to discontiguous physical memory.
> That's what I thought. The address your user-space program uses and the actual address in memory are two different things, aren't they?
Right. Userspace uses virtual addresses, which get mapped through page tables to become physical addresses. Some kernel addresses get identity-mapped (meaning that the virtual address matches the physical address), while some kernel addresses go through translation as well. (Specifically, on Linux, kmalloc allocates identity-mapped addresses, while vmalloc allocates virtual addresses; vmalloc allows you to have a large virtually contiguous buffer without requiring a large physically contiguous region of free memory.)
> Isn't this what enables the operating system to scramble the allocations it gives you to make it harder to implement a buffer-overflow attack?
You can do Address Space Layout Randomization (ASLR) for either virtual or physical address spaces; you want to do it for whatever type of address an attacker could otherwise make use of. Userspace processes can use ASLR for their virtual address space, so that attackers can't make use of fixed addresses. The Linux kernel doesn't normally map virtual pages to any particular well-known location in physical address space, either. The kernel can also use ASLR for some (though not all) of its own kernel-space addresses. Beyond that, recent versions of Linux also try to avoid exposing kernel-space addresses to non-root users.
Also note that ASLR doesn't generally introduce enough randomness in a 32-bit address space.
The fix for both this issue and the pinning issue is big changes to the garbage collector. Since the language and standard library have stabilised with Go1, changes to the runtime are going to be the focus of development so this issue will probably get a fix.