So I’d go with no at the moment because I can easily get an LLM to contradict itself repeatedly in increcibly obvious ways.

I had a long ass post but I think it comes down to that we don’t know what conciousness or self awareness even are and just kind of collectively agree upon it when we think we see it, sort of like how morality is pretty much a mutable group consensus.

The only way I think we could be truly sure would be to stick it in a simulated environment and see how it reacts over a few thousand simulated years to figure out wether its one of the following:

• Chinese room: The potential AI in question just keeps dying because despite seeming intelligent when prompted with training data it has no ability to function when its not spoon-fed the required information in advance. (I think current LLMs are here given my initial statement in this post).
• Animal: It survives but never really advances beyond figuring out the behaviours required for survival, its certainly concious at this point but works more like a dog where it can follow commands and carry out tasks but has no true understanding of the meaning behind them.
• Person: It starts seeking out information in ways not immediately neccesary for its survival and basically does what we did with the whole tool thing and speculative reasoning skills, if it invents an equivelent to writing then we can be pretty damn certain its human level and not more like corvids (tools) or ants (agriculture)

Now personally I think that test is likely impractical so we’re probably going to default to its concious when it can convince the majority of people that its concious for a sustained period… So I guess it has free will when it can start or at least spark a large grass roots civil rights movement?

I’d say it ends when you can’t predict with 100% accuracy 100% of the time how an entity will react to a given stimuli. With current LLMs if I run it with the same input it will always do the same thing. And I mean really the same input not putting the same prompt into chat GPT twice and getting different results because there’s an additional random number generator I don’t have access too.