I've played around with ChatGPT a bit over the last few days. I'm wondering how it might be useful for Daystream. I can't get past the basic inaccuracies in the system, though.
For example, today I asked it to "Describe some events or happenings that occurred on March 29, 1983." I thought API calls using a prompt like that might be an interesting way to generate user-independent content for historical days. I'm not sure why I picked 1983, but 40 years ago today seemed like a decent test. I was 13.
ChatGPT quickly gave me a list of seven events that, according to the AI, "occurred on March 29, 1983." The first event on the list caught my eye because I have specific memories of it: "The final episode of the television series 'MAS*H' aired on CBS, drawing a record-breaking 125 million viewers in the United States."
Unfortunately, ChatGPT got this one wrong. Basic web research using Wikipedia and IMDB reveals that the final episode of MASH, Goodbye, Farewell and Amen, actually aired on February 28, 1983, 40 years and 1 month ago.
This is an easy one to get right, too. The last episode of MASH is generally considered to be one of the most-watched scheduled television episodes of all time. It's the GOAT of episodic television. If AI got details wrong on something that is so easy to verify, what might it get wrong on things that aren't as easy to check? Or that can't be checked? What about a historical event or item that has such a low level of general interest that people only touch its details once a generation, once every other generation, or less? Human verification of AI generated historical content isn't just something that should be done, it's something that must be done to avoid a quiet rewriting of the details of human history. Anything that isn't human-verified should be labeled as such, and treated accordingly.
ChatGPT, and AI generation of content generally, still intrigues me and I think there might be a place for it in Daystream at some point. But, in light of errors like this that are revealed with basic fact-checking, I currently have no confidence in using it to assert that something actually happened on a particular day in the past, or that a list of various things occurred on a particular day.
postposted by matt in Wednesday, March 29, 2023