When Dimitris Papailiopoulos first asked ChatGPT to interpret colors in images, he was thinking about “the dress”—the notoriously confusing optical-illusion photograph that took the Internet by storm in 2015. Papailiopoulos, an associate professor of computer engineering at the University of Wisconsin–Madison, studies the type of artificial intelligence that underlies chatbots such as OpenAI’s ChatGPT and Google’s Gemini. He was curious about how these AI models might respond to illusions that trick the human brain.
The human visual system is adapted to perceive objects as having consistent colors so that we can still recognize items in different lighting conditions. To our eyes, a leaf looks green in bright midday and in an orange sunset—even though the leaf reflects different light wavelengths as the day progresses. This adaptation has given our brain all sorts of nifty ways to see false colors, and many of these lead to familiar optical illusions, such as checkerboards that seem consistently patterned (but aren’t) when shadowed by cylinders—or objects such as Coca-Cola cans that falsely appear in their familiar colors when layered with distorting stripes.
In a series of tests, Papailiopoulos observed that GPT-4V (a recent version of ChatGPT) seems to fall for many the same visual deceptions that fool people. The chatbot’s answers often match human perception—by not identifying the actual color of the pixels in an image but describing the same color that a person likely would. That was even true with photographs that Papailiopoulos created, such as one of sashimi that still looks pink despite a blue filter. This particular image, an example of what’s known as a color-constancy illusion, hadn’t previously been posted online and therefore could not have been included in any AI chatbot’s training data.
To read more, click here.