What’s going on here is that the input text is passed through a tokenizer that turns the whole word into a vector before it’s passed to a neural net. The inner neural net can’t even see the character-level details so it’s forced to guess. This also accounts for challenges with getting it to rhyme.
