There are cases where the network can reproduce code verbatim, just like there are some functions that a good programmer may know by heart. However, this is not how Copilot normally works; it acquires understanding of code, not a copy-paste library.
Well, that's overstating things: it's kinda an open question under what circumstances GPT will learn text by heart. However, it seems to be extremely rare.
(Human creators also sometimes reproduce things they've read verbatim, without noticing.)
Would you consider a Markov chain to have an understanding of the input data? It too sometimes reproduces training data verbatim, but mostly produce remixes of many short snippets instead.
On the spectrum from Markov chain to Human comprehension, I'd submit that GPT is much closer to the former than the latter.
I would consider a Markov chain to have "the sort of thing that understanding is a more complex version of" of the input data. Markov chains compress inputs. Understanding is compression. So I'm willing to bite that bullet.
I submit that GPT is much closer to the latter than the former, and may even overtake it in its domain. It lacks hardware that the human brain uses to reflect on its knowledge and understand what it is thinking - but just as an "overtuned intuition", just like how AlphaZero's network forms a quite capable chess AI without even using tree search, just by "taking the move that feels right to it", I think it's on par with human intuition, or even stronger, because it cannot rely on reflection like humans can.
(In other words, I think there isn't a spectrum of Markov chains to human comprehension. Human comprehension is a different kind of system. However, there is a spectrum of Markov chains to human intuition.)
Well, that's overstating things: it's kinda an open question under what circumstances GPT will learn text by heart. However, it seems to be extremely rare.
(Human creators also sometimes reproduce things they've read verbatim, without noticing.)