Self-sufficient, autonomous AI writing its own AI software script. THAT would send shivers up my spine.
Something like Douglas Lenat's
Eurisko? That program also became world champion in quite a few (obscure) fields, including
one game where they kept changing the rules in order to prevent Eurisko from taking advantage of loopholes it found, and Eurisko kept finding new ways to win the game.
And Eurisko was written as a generalisation of
AM (1976), a program that from basic set theory discovered numbers, algebra (including the unique prime factorisation theorem) - partly to counter criticism about the opaque nature of the AM innards. His current ongoing project is Cyc, which is about representing all the knowledge as bright humans have and more - including all the vague and informal, redundant, and/or contradictory facts that we carry with us.
Maybe interesting for this forum is that, based on his experience of seeing evolution in action within Eurisko (as opposed to biologists who haven't seen much evolution, and certainly not inspectable in detail), Lenat has intelligently critiqued standard (neo-)Darwinism.
Unfortunately I am separated by more than 10 000 km from my books and articles (which are boxed and stored in a container anyway :-) - so I cannot provide you with all the juicy details.