- An LLM has more knowledge, but it doesn't have information about what you specifically want to convey. Anything you give to the LLM may be good information. Anything the LLM adds is not additional real information, because whatever it adds can be inferred based on whatever you wrote. Or the LLM adds additional information that can't be inferred based on what you wrote, which is even worse because that's basically just misinformation.
- If you're giving additional prompts to the LLM to refine its output, then you're the one adding real information, not the LLM. The LLM is just rephrasing the information and adding noise.
- That's outside the scope of what I claim. The claim was that if you give an LLM 300-bits of information it cannot add more things that are relevant that the reader could not also do themselves. It certainly can. The LLM may add information you do not want to convey. Which is easily mediated by removing it or additional prompting.
- You are adding real information. The LLM is also adding real information. That's the entire point. It happens very often that an LLM suggest something to me that I did not know or simply did not think about. An LLM solved Navier-Stokes recently. That was most certainly not just adding noise. That's real information purely generated by an LLM. Information that was worth a million dollar price. Information that man centuries of mathematicians were not able to do generate.
Keep in mind that I'm using "information" in the information-theoretic sense. One could argue that because that particular solution to NS is provable, it is therefore implied by the propositions they started with and adds no new information.
And I'd still rather read a human's interpretation of the solution to NS than read whatever the LLM wrote.
- If you're giving additional prompts to the LLM to refine its output, then you're the one adding real information, not the LLM. The LLM is just rephrasing the information and adding noise.