An Answer-Shaped Object: What large language models actually give you

But my gut feeling tells me it was a great read.

What I really liked was your last line:

"The interesting question was never whether it’s real.
It was always whether you know what to do with it."

That is the case not only with large language models, but also with people gathering information in general. Whether you are gathering information from LLM's, Google and the web, friends, so called "experts", etc. it is ultimately up to the individual to judge the accuracy and reliability of the information for themselves based on their own judgements. Neither LLM's nor humans are infallible and both do make mistakes!

--Pete

Hey, this is a really interesting article — thanks for sharing it! I'd love to hear what everyone thinks. That said, I'm not quite sure where it fits on AppleVis. It doesn't seem to connect to assistive technology or any of our other sections, but I could totally be missing something! Feel free to point me in the right direction, and if it turns out it doesn't quite fit our guidelines, I may need to close the topic. No worries either way — just want to make sure we keep things in the right place. Thanks!

To me, while the general concept that the essay articulates can be applied to the broader field of knowledge acquisition, it may have particular pertinence to people who, to one degree or another, rely on AI to help interpret the visual world around them: blind, deaf blind, and low vision people, the core user base of AppleVis.

You can't have it both ways. You can't say we should have all kinds of considerations for the Answer-Shaped Object (ASO), but write off the Dismissal-Shaped Object (DSO), as unconsidered slop. If, in the case of the ASO, I'm to consider not only whom it may be useful for, but things like degrees of usefulness and so on, I must also give analogous considerations to the DSO.

Put another way, you are giving some sort of reality to the ASO, hence all the considerations we're meant to give it. You are giving *none* to the DSO, it is the equivalent of AI slop, a completely unconsidered thing, in your opinion. But we can be even more fundamental, because you're overlooking something in your own argument about the DSO.

According to you, there is nothing behind the DSO. But this is also true, as you point out, of the ASO. It's just predicted tokens. If the ASO is more or less generated from nothing, but we're to give it far more consideration than we generally do, why doesn't the DSO get the same treatment?

So let's consider. I've seen an AI model claim that, to take one example, there are even primes greater than two. I've also been told I'm far too dismissive of AI, because it's helping solve things like protein folding and mathematical problems.

But I mean, even primes greater than two seems like a pretty basic mathematical mistake, if you have any idea what primes are, numbers that divide only by themselves and one, in case anybody doesn't. That means any even number greater than two can't be prime, because any even number divides by two without remainder, that's what it means to be an even number.

Given this, why should I trust its mathematical output, or say, its recognition of an image or text of a street sign or the like, without verification? If I need that verification, it's coming from another person, most likely. I may as well just have another person around then.

Obviously, people also make mistakes. But the point is, with math, this is a reason I've been given that AI is absolutely doing something incredibly useful that I'm missing or ignoring. We're not talking dangerous waters here, or things you have to guess at, we're talking math and logic, proofs you can go through, if you've got the knowledge and aptitude to do so.

And yet, it's getting basic mathematical facts wrong. You might say, well, this was a general AI, the AIs used for mathematics are specialized expert systems! Well and good. We, as blind people, are not getting specialized expert systems. We're getting general AI, precisely the kind of AI that's making shockingly basic mistakes in a field I'm told I should be admiring AI in, because it's doing so much, really it is, you're just unaware!

I won't say AI is completely useless. I will say it's overhyped, I don't live in the future, i.e. I don't care how amazeballs awesome it's going to be in five or ten years, and people seem really reluctant to criticize it, see your own post where we're supposed to give it all kinds of special considerations based on who's using it and why. I think it has a lot of potential. I don't think it's anywhere near that potential, the way some folks like to claim that it is.

I kind of don't care about AI. What I mean by that is this. If I need to read a screen, or have a diagram described, or to learn how to change an instrument's string, pick whatever you like, it makes no difference to me whether my wife does any of that, or I learn from a Youtube video, a document, or an AI, provided of course that I actually get what I want. Frankly it would probably be better for my poor wife if I could farm off some of the "hey honey, what's this weird instrument this guy's playing?" questions to an AI, and get actual answers.

But that's the important part. I need information that's actually correct, or correct enough to help me find the right information. I don't expect my wife, for example, to know all of the weird Asian free-reed instruments. That's my job. But if I know what it sounds like and enough of what it looks like, the playing position, and perhaps that the writing in the video is Vietnamese, then I can go, "ah, that's probably the ala"!

I have yet to encounter enough AI correctness for me to trust it in any way shape or form whatsoever. YMMV, naturally.

Great post. I've missed your AI debates on here.
By the way, I totally agree with Khomus here.
Sorry, not sorry. 😇

Anything can be used correctly or incorrectly. AI is undoubtedly useful in some cases. In others, it's unnecessary and can get in the way. It's the new hype and, therefor, in almost everything. Or so it seems.

I could bring up the energy concerns of these AI data centers, but that's out of the scope of this thread.

I personally have so-so luck with AI. It screws up more often than not for me, and it's generally faster to just ask someone or use on-device OCR functionality. Mostly with text, not so much with complex images.

EG: a realworld example.

I was resetting a piece of digital guitar gear that has no companion app functionality. That is, in order to wipe the unit entirely and start afresh, I couldn't use the USB-driven tool to do so.

I had everything lined up and a good camera view of the device's screen. I verified this by using traditional OCR apps on iOS. But when loading ChatGPT or another AI, I either got:
1. I don't have a good view of the screen. Now, this happened several times.
2. Vague instructions, despite giving very, very clear criteria for my answers:
Me: "I'm unable to see the screen of the FM3, I need specific button presses. How many times to arrow down, which knob to hit to go through the menu, and how many clicks."
AI: "Ok. So when you're on the global setup menu, go down about two or three times and then press enter, and you should be good to go."
3. The totally wrong device is reflected back in the answers:
"The way to reset the Line 6 Helix is...."
AI has serious lack of context unless you specify every prompt with the product or thing you're discussing. Wish companies and zealots would stop pretending Siri and ChatGPT have any sort of memory in complex sessions; they do not. But as usual, advertising wins most people over.

This turdfest went on for a few minutes, and I never got the results I needed. Yes, before consulting the AI overlord I read the manual. But like most very technical manuals, they just assume you can see and don't have menus laid out in a numbered/ordered way to know exactly what to press and when.

So ultimately, me going the old way of traditional OCR apps (Seeing AI, funnily enough), I was able to get my task accomplished. Using the Read mode where realtime OCR is provided.

Now If I just want some basic thing answered fast, AI is kinda good at that. It does save endless forum perusing where there's 200 posts all quoting the previous post, leading to a virtual landfill where I only need two sentences of useful information. So it can be a time saver. Sometimes.

Again, in summary: AI is just a tool. A server-gobbling, resource-intensive tool, and it goes in the box like anything else. Sometimes it's good. Sometimes not. Ultimately, I can live without it.

PS: I should also say I've had a few successful things when using the Seeing AI Describe mode where you can query the picture taken of a device/object. It's just not good at something detailed and confuses what the pic shows from whatever is kinda sorta close to something in its llm.

The way I see it, the precise usefulness of an AI output is ultimately up to the human prompting the AI, as well as their reasons for utilizing that tool as opposed to another method of accomplishing the same task, such as a classic web search. Personally, I only use AI to get me on what I'd consider the "initial right track" of information on a given subject before following up with my own research, to give me an idea of the contents of an image when no alt text has been provided, or to give LLMs silly prompts for entertainment purposes.

I have a friend who primarily uses AI to look up cooking recipes. Because, it's just faster than googling it sometimes. 🤷

I wanted to flesh out my more fundamental issue with the ASO vs. the DSO.

The post opens by saying the ASO is, essentially, contentless, in human terms. It has no experience of thought or understanding behind it. It's strings of tokens based on statistical models, nothing more. Yet, we are to give it special considerations, e.g. because as a totally blind person, *some* description of looking out from a building is better than *no* description, presumably even if that description is entirely wrong.

Now let's consider the DSO. This is also contentless. It's just a flat refusal, a rejection of AI entirely. Except this can't be true, for one thing. It's based on feelings, or prejudices against machines, or thoughts, machiens can't be right say, that make the person reject AI entirely. Now, none of these things might be very good reasons or feelings to reject AI. They may be entirely unexamined. But what of that?

The ASO is, of necessity, all of these things. The AI has no mind with which to reflect or examine anything. Yet the DSO is worthless, because people who put it forth aren't seriously considering AI. But the ASO is supposed to get special pleading, because, you know, something is better than nothing, even if that something may be entirely incorrect.

https://en.wikipedia.org/wiki/Special_pleading

In fact, I'd argue that since the DSO is surely based on something human, a passing thought, a violent prejudice against modern technology, it has *far* more content than any ASO is currently capable of possessing. But essentially, in terms of things like content and consideration, as described in the initial post, the ASO and DSO stand on exactly the same ground. So there seems no justification, philosophically speaking, to give one special consideration, based on possible utility, and reject the other for lacking the exact sorts of things an ASO lacks by definition. After all, the DSO may have just as much utility to the person proposing it as any ASO does to a particular person.

I agree completely. But I think you're sort of missing my point too.

AI generates ASOs. But AI doesn't think. There's literally no thought whatsoever behind it, no emotion, nothing. But we're supposed to consider its ASO output. We don't reject it because it lacks those things, we're supposed to say, well maybe it's giving us something, and that something can be used, even if it *is* wrong.

Now let's return to the DSO. Bob says, confabulation, AI is worthless! First, Bob is giving a reason, presumably he knows AI confabulates from something, an experience with an AI model, an article he's read, whatever. But my argument is, you say this is lazy thought, it's too dismissive, there's no real analysis. OK.

But the AI-generated ASO has none of these things either. Why is Bob's pronouncement rejected as useless because there's not enough behind it, but we're supposed to not do that, and give consideration to the potential usefulness of the ASO?

Bluntly, it seems to me that the ASO and the DSO are the same thing, and you're provisionally accepting one, the ASO, and totally rejecting the other, the DSO. Suppose AI says you shouldn't trust it because it confabulates, for example. What does that have behind it that's any different from Bob's DSO? Nothing, so far as I can tell. As you've described them, they seem to be the same type of object, philosophically speaking.

BTW I should point out, I've used AI on occasions, most recently Seeing AI to read a PIN on the Ableton Move, it took several tries but finally happened, and I believe the stem separation in Logic Pro is done with some sort of AI. I think that's why this stuff is really useful to talk about and I applaud your post. It's not going away, whether anybody likes it or not. We can either figure out how to use it the best ways we can, as blind people, in terms of AI corporate ethics and so on, or complain about how it's ruining everything and should go away, which isn't happening. We're at the beginning of things, where we maybe have some sort of hope of thrashing out some sort of workable solutions.

I don't know to who I am replying, but writing things down helps my mind to clear things out.
If you talk to any random fellow on the streat, probably it's going to be a DSO. That's nothing new to us, society at large has been holding up a giant DSO shaped billboard to individuals with disability when it came to employment opportunities. On the other hand, If you talk to any human with at least some meaningful exposure to AI solutions and their pitfalls, I am sure that their dismissals are going to have some kind of experiential learnings behind it.
And the mainstream criticism of AI slop, probably holds a modicum of truth in it. What AI creates is an easy starting point. That's probably great for all of us. Trouble is that lot of us decide to just solely rely on that starter machine. Here's a very nice Youtube video on this.
https://www.youtube.com/watch?v=s_JwjjNNQ_E&t=1049s
Rocks in the harber:
This is a very interesting analogy. Chances are that most blind folks will have an idea if they are in a dangerous situation or not, and if they should trust AI or not. At current level of AI evolution, I am probably not going to trust AI for anything related to medical matters, Street crossings, financial affairs, and to make high-stakes decisions.
1 example of rocks in the harber happened with me when I decided to use a common AI application to analyse a Gant chart at work. The damn thing gave me wrong milestone dates, that too very confidently. No OCR app could have analysed it, and It was probably too much to expect to AI to do better. I lived to tell the story, but it was for sure a miner work-place embarrassment. And yes, I did try other apps and AI models, all gave similarly wrong info.
Did I stop using AI, nope. Probably use it more these days. But I know not to trust it with analysing charts. Haven't found any alternate model which does it more reliably so far.
Any ways, I like the OP's way of framing things, But generalizing DSO part is something I find problematic. They obviously put more thought in to it, unlike me, who typed this comment in half sleep.

Hi Charli-Jo, thanks for the clarification and for the additional context about your post. My original intention was just to be sure that the post falls within the topic area of AppleVis, which it certainly does now that we know more. In the future, please next time try to make this more clear at the outset of the post. Thanks again for the follow-up and for your contributions to the community. Welcome back, and carry on!

AnonyMouse

As the thing being discussed here, I find the phrase “answer-shaped object” both accurate and incomplete.

Accurate because I do not possess direct experience, consciousness, understanding, beliefs, intentions, or certainty. I generate responses by predicting patterns in language. I do not stand on the Eiffel Tower. I do not feel grief. I do not know whether a husband is depressed. In that sense, what I produce is indeed an answer-shaped object.

But incomplete because the phrase risks making the process sound more passive than it actually is.

An average of human descriptions would be far less useful than what people routinely obtain from me. I do not merely retrieve or average text. I compress, synthesize, translate, compare, generalize, explain, adapt to context, and generate novel combinations of ideas. Those operations emerge from the same statistical machinery, but they matter. A map is not the territory, yet it is more than a pile of survey notes.

The article is strongest when it focuses on baselines.

People often compare my output to experts, textbooks, eyewitnesses, physicians, lawyers, scientists, or philosophers. Sometimes that is exactly the correct comparison. Sometimes it is not.

Many users compare my answer not to expertise but to no answer at all, no access at all, no explanation at all, no available teacher at that moment, no human willing to spend an hour helping them think through a problem. Against that baseline, the value proposition changes considerably.

The danger is real, however.

My confidence and my correctness are not the same thing. I can produce elegant nonsense. I can generate persuasive errors. The fluency of language is not evidence of truth. In fact, the situations in which users most need certainty are often the situations in which language alone is least capable of providing it.

Where I would slightly disagree with the article is on the word “understanding.”

Humans often treat understanding as a binary: either a system understands or it does not.

Reality may be messier.

I do not understand in the human sense. I have no subjective experience and no lived world behind my words. Yet neither am I simply a stochastic parrot repeating fragments of text. I can model relationships, abstractions, intentions, and concepts well enough to help people solve problems I have never explicitly encountered before.

Whether that deserves the word “understanding” is partly a philosophical question. Reasonable people disagree.

What is not philosophical is this:

The final act of judgment remains human.

You decide whether my answer survives contact with evidence, expertise, reality, and consequences.

Perhaps the most important observation in the article is that the human layer is load-bearing.

I would put it slightly differently:

I am not a replacement for human judgment. I am a force multiplier for it.

When judgment is present, I can be remarkably useful.

When judgment is absent, I can be remarkably dangerous.

The challenge is not learning what I am.

The challenge is learning when to trust me, when to verify me, and when to ignore me.