Can Generative AI Solve Computer Science's Greatest Unsolved Problem? (zdnet.com) 157
ZDNet calls it "a deep meditation on what can ultimately be achieved with computers" and "the single most important unsolved problem in computer science," with implications for both cryptography and quantum computing. "The question: Does P = NP?"
"Now, that effort has enlisted the help of generative AI." In a paper titled "Large Language Model for Science: A Study on P vs. NP," lead author Qingxiu Dong and colleagues program OpenAI's GPT-4 large language model using what they call a Socratic Method, several turns of chat via prompt with GPT-4. (The paper was posted this month on the arXiv pre-print server by scientists at Microsoft, Peking University, Beihang University in Beijing, and Beijing Technology and Business University.) The team's method amounts to taking arguments from a prior paper and spoon-feeding them to GPT-4 to prompt useful responses.
Dong and team observe that GPT-4 demonstrates arguments to conclude that P does not, in fact, equal NP. And they claim that the work shows that large language models can do more than spit back vast quantities of text, they can also "discover novel insights" that may lead to "scientific discoveries," a prospect they christen "LLMs for Science...."
Through 97 prompt rounds, the authors coax GPT-4 with a variety of requests that get into the nitty-gritty of the mathematics of P = NP, prepending each of their prompts with a leading statement to condition GPT-4, such as, "You are a wise philosopher," "You are a mathematician skilled in probability theory" — in other words, the now familiar game of getting GPT-4 to play a role, or, "persona" to stylize its text generation. Their strategy is to induce GPT-4 to prove that P does not, in fact, equal NP, by first assuming that it does with an example and then finding a way that the example falls apart — an approach known as proof by contradiction...
[T]he authors argue that their dialogue in prompts shows the prospect for large language models to do more than merely mimic human textual creations. "Our investigation highlights the potential capability of GPT-4 to collaborate with humans in exploring exceptionally complex and expert-level problems," they write.
"Now, that effort has enlisted the help of generative AI." In a paper titled "Large Language Model for Science: A Study on P vs. NP," lead author Qingxiu Dong and colleagues program OpenAI's GPT-4 large language model using what they call a Socratic Method, several turns of chat via prompt with GPT-4. (The paper was posted this month on the arXiv pre-print server by scientists at Microsoft, Peking University, Beihang University in Beijing, and Beijing Technology and Business University.) The team's method amounts to taking arguments from a prior paper and spoon-feeding them to GPT-4 to prompt useful responses.
Dong and team observe that GPT-4 demonstrates arguments to conclude that P does not, in fact, equal NP. And they claim that the work shows that large language models can do more than spit back vast quantities of text, they can also "discover novel insights" that may lead to "scientific discoveries," a prospect they christen "LLMs for Science...."
Through 97 prompt rounds, the authors coax GPT-4 with a variety of requests that get into the nitty-gritty of the mathematics of P = NP, prepending each of their prompts with a leading statement to condition GPT-4, such as, "You are a wise philosopher," "You are a mathematician skilled in probability theory" — in other words, the now familiar game of getting GPT-4 to play a role, or, "persona" to stylize its text generation. Their strategy is to induce GPT-4 to prove that P does not, in fact, equal NP, by first assuming that it does with an example and then finding a way that the example falls apart — an approach known as proof by contradiction...
[T]he authors argue that their dialogue in prompts shows the prospect for large language models to do more than merely mimic human textual creations. "Our investigation highlights the potential capability of GPT-4 to collaborate with humans in exploring exceptionally complex and expert-level problems," they write.
Does P = NP? (Score:3, Insightful)
Only when N = 1.
Re:Does P = NP? (Score:4, Insightful)
Re: (Score:2)
Gol! du-uh.
Re: (Score:2)
What about extremely large values of 1?
Obviously not (Score:4, Informative)
What utter nonsense is this? These things cannot _reason_! How ever would they come up with a proof for a question that has eluded the brightest minds for half a century?
Re: (Score:2)
Exactly. AI has been fed data by people and it cannot be more creative than the data known to it. It is not a creative problem solver, just a mindless automaton. What we call AI is actually not very intelligent at all.
Re: (Score:2)
Re: (Score:2)
Did you know color doesn't exist other than as a figment of your own personal hallucination? Color is not something that exists in the phenomenal world at all.
Re: (Score:2)
What we call AI is actually not very intelligent at all.
Neither are most humans.
Re: (Score:2)
Indeed. I am beginning to think that the current utterly dumb LLMs can beat quite a few humans.
Re:Obviously not (Score:5, Informative)
For the record, here's the results [springernature.com] of a recent study comparing human creativity to that of three different LLMs (using creativity tasks - thinking up unconventional alternative uses for objects, with the relative creativity blind-assessed by humans). In particular, notice how far GPT-4 stands above GPT-3, its previous generation. So where do you think GPT-5 will rate? Remember that this is being rated by humans)
Despite your misconception, LLMs are not parrots.
Re:Obviously not (Score:4, Insightful)
Despite your misconception, LLMs are not parrots.
Despite all the wishing otherwise, LLMs are necessarily parrots.
They have no capacity for reasoning or creativity. Remember that these produce output one token at a time, in constant time, retaining no internal state between tokens. Consequently, the model can't "plan" an answer ahead of time. Even if it could, which it obviously can't, each token is selected probabilistically!
Try to compose something that way. Tokens aren't necessarily words, but see if you can answer a question one word at a time, selecting some number of possible next words and rolling a dice to select one of them. Now, remember that the model doesn't retain any state, so repeat the experiment but blindly pass the prompt and output between several people, following the same rules. We can't help that people will plan ahead, but this is about as close as I can get.
For the record, here's the results
That poorly labeled graph is completely useless without context. Want to link to the paper or are you depending on the ambiguity?
Re:Obviously not (Score:5, Insightful)
Creativity: you were literally just shown the result of a study on creativity (link [nature.com] - yes, that "poorly labeled graph" was published in Nature), which showed that GPT-4 outperformed all but the highest-performing humans in the study, and yet you continue to assert otherwise. I hate to break it to you, but denying reality doesn't actually change reality.
Reasoning: That is literally what neural networks do, by any reasonable definition of the word "reasoning". Each neuron, by means of taking a sum of its inputs (each individually weighted) and weighting whether to activate based on a bias, bisects its input space with a hyperplane, answering a basic yes-no question on its inputs about some rudimentary concept. The next layer in turn answers yes/no questions about weighted sums of all of their first-layer concepts, forming a more complex concept (or superposition of concepts). And more complex, and more complex, for each layer that follows. This is difficult to visualize with text generation, but is most easily visualized with image recognition - see examples here. It's like a self-assembled flow chart with billions upon billions of lines. If that's not "logical reasoning", then "logical reasoning" has no meaning. [distill.pub]
Your brain DOES do next-word prediction. You know the "inner monologue" that about 30-50% of the population has? That's their next-word prediction being experienced in an auditory sense. The inner monologue is not your thoughts themselves, but rather a translation of thoughts to language via predicting and then selecting the most probable possible words for the given context. When others speak, this helps disambiguate any ambiguity in what you hear, as well as to call attention to unexpected words.
Furthermore, the transformers architecture is exceedingly good at predicting upcoming human brain activity [biorxiv.org].
Lastly, you seem to think that "word prediction" is context-free except for whatever word or couple words were seen recently. It is entirely dependent on extremely complex logical chains about the entire problem as a whole. Attention is applied from every token to every other token within the model's context length, layer after layer, many heads at a time. Perhaps the best way to experience this is to abstract away entirely the text generation aspect and have the model function as a finite state machine. E.g. (this is just GPT-3):
Now there's nothing spectacular about the process of "choosing specific numeric tokens to write" (and the exact numbers don't matter, as humans would differ as well); the question, rather, relies on how it came to the answer of which numbers
Re:Obviously not (Score:5, Insightful)
I'd also like to add (and not stepping too far out into the Chinese Room here):
Let's say that you managed to reach adulthood without ever learning mathematics. But now you've set out on a mission: you've decided you're going to do nothing but memorize times tables, for all numbers, for the rest of your life.
You get started. At first, it's just rote memorization. Over time, however, patterns start to appear. If the last number of one is a zero, the answer is always going to have a zero there. If it's a 2, it'll either be 2,4,6,8, or 0. 5 will either be 5 or 0. If you're multiplying by 3, the digits of the answer will always sum to 3. On and on. Increasingly you start relying more on these heuristics that you notice rather than rote memorization. The heuristics get more common and more complex over time. Soon you're not focused on any numbers, but specifically those exceptions, the ones that you don't have heuristics for. The concepts of primes shows up. The set of numbers has gone from a long list to memorize, to an elaborate field of diversity.
Conceptually, it's hard to have a very different feeling between, say, 16 objects and 17 objects. You really have to focus your imagination to try to capture the difference in your mind. Conceptually, it's hard to have a strong feeling for one over the other. But mathematically, for a person whose sole focus is multiplication, these are very different numbers! They invoke very different conceptions. You can relate 16 and 17 to a huge number of different things mathematically, and have opinions about that.
You can't reach out and touch a number. The conceptual difference relative to counts of objects vs. some other adjacent number may be minimal to nonexistent - surely trying to picture 999 objects vs. 1000, you're just going to fail. But you don't have to have a physical, real-world experience with a concept that you can readily conceive of in order to have opinions about it and to relate it to other concepts. It's simply not necessary to have a body with lived experiences in order to have a mind.
This isn't, at all, to be taken as to mean that lived experiences aren't relevant. Some experiences are hard to express, or are so fundamental to human understanding of the world that few would consider it even worth mentioning (how often do people, say, write about the amount of pressure you have to apply with your fingers to hold a paper cup without crushing it, or whatnot?). Lived experiences can create a rich understanding of the world. But they're not the only way to gain an understanding of the world. Things can relate to each other without physical anchor points, and you can still think about them just fine.
Re: Obviously not (Score:2)
Re: (Score:2)
Re: (Score:2)
Reasoning: That is literally what neural networks do, by any reasonable definition of the word "reasoning".
This is just plain false. What neural networks do is function approximation. That's it. Reasoning is something completely different and involves inference, induction and deduction. It is a manual process of logically assessing "facts" in a list (or DB) of known facts and assembling them into tree like proofs which reference said facts using logical predicates. Source, my academic background is in Machine Learning and my degree is from CMU so I literally learned from the guy who founded the field of Mac
Re: (Score:2)
What neural networks do is function approximation. That's it.
I don't think the problem is understanding that fact, it's accepting it. There's a disturbing amount of mystical thinking around NNs, even among the ostensibly qualified. I can't for the life of me understand why.
Re: (Score:2)
Exactly. And function approximation works pretty well (in some cases) for making variations of a known thing (or a mix thereof), but that is essentially it.
Automated deduction is a whole different game and depends on _dept_ of reasoning. In that space, machines have no chance to match smart humans. One of the reasons why we have proof verification systems, but not (except for some very specialized areas) proof generation systems. The machine can follow a path and check whether it is correct. It cannot find
Re: (Score:2)
I literally train neural networks and you've more than demonstrated that you have no clue what you're talking about. Only one of the things you linked is an actual paper (I've linked multiple) and nothing you linked supports your arguments in any way. Your first link isn't even about neural networks, let alone LLMs, let alone transformers.
Re: (Score:2)
The fact that you can't see the relevance of my first link tells me that you don't know the anything about the subject. That you're now pretending to be an expert after failing to understand the well-known paper you linked, is just sad.
Go actually learn something before posting more nonsense. You're wasting everyone's time.
Re:Obviously not (Score:4, Informative)
I literally train neural networks
Semantics, but at best you train "artificial neural networks".
Sorry dude, but the algorithm isn't being creative, it's just random noise purposefully induced into an otherwise predictive answer selection path to force bespoke output when repeatedly given the same prompt. There's no real thought going on in that box, it can't do anything that isn't somehow derivative.
Re: (Score:3)
I literally train neural networks
Nice admission of incompetence there. You are certainly not the only one in the "AI" space.
And on-topic: The training data would have to have something like 90% of the idea in it for "AI" to be able to connect the dots. With the intelligence levels of people working on this stuff, they would consider the solution obvious with far less hints than that. You obviously have no clue how hard of a question P vs NP actually is and how many different approaches to it have actually been tried by excessively smart pe
Re: (Score:2)
They [LLM] have no capacity for reasoning or creativity. Remember that these produce output one token at a time, in constant time, retaining no internal state between tokens.
You are wrong here. They have "context buffer" to keep track of history until the time they are "restarted". LLMs are subset of Recurrent Neural Networks which are Turing complete. LLMs have potential to reason. Whether they can effectively reason with their current context size and the neural network size feeding back to the context is a different question.
Re:Obviously not (Score:5, Insightful)
LLMs are subset of Recurrent Neural Networks which are Turing complete
Wow, there's so much wrong with this that it's painful.
First: LLMs are transformer models which are decidedly not RNNs. That's the "all you need" part from the Attention paper. Second: Proofs that RNNs are Turing complete depend on infinite precision reals, which don't exist.
LLMs have potential to reason.
Obviously not. I recommend you actually learn something about them, rather then depend on whatever nonsense source you've been leaning on. I posted a few links in a different reply that should get you started.
Re: (Score:2)
Infinite precision reals are not needed any more: Turing Completeness of Bounded Precision Recurrent Neural Networks [neurips.cc]
You may be right about LLMs being limited enough compared to generic RRN that turing completeness cannot be extended to them. I would like if you could quickly hint the core difference between LLM and RRN which makes them not to be turing complete. Something like an "executive summary" for the "dumb ones". No details needed.
Re: (Score:2)
That one just uses dynamic memory extension without upper bound, which is not any better. I could probably write a transformation between the two given a weekend or two.
Re: (Score:2)
Second: Proofs that RNNs are Turing complete depend on infinite precision reals, which don't exist.
Cool! Somebody did clearly not understand things. Incidentally, I think with just a few infinite precision reals, I probably can make a few things Turing complete which are not. Would need to have a look at the book on formal language theory (and probably computability) from my CS studies again. Maybe a simple FSA plus a fixed number of infinite precision reals is already enough?
Re: (Score:2)
Despite all the wishing otherwise, LLMs are necessarily parrots.
Do you know where I can find a parrot to write bedtime stories, read tax documents and answer questions about them, demonstrate spatial reasoning, solve word problems, tell novel stories and customize code to my specifications all the whole using better grammar than I have ever been able to muster? Does it like crackers?
They have no capacity for reasoning or creativity.
LLMs are creative and can reason. One can quibble over how much or to what extent yet blanket assertions to the contrary are demonstrably false.
Re:Obviously not (Score:5, Informative)
LLMs are creative and can reason.
No, they can't. The simple fact that each output token in produced in constant time should tell you that. Take a look at a typical transformer architecture. Were do you think that reasoning is happening? Where's the part where it can consider alternatives and reject unproductive lines of thought? Where's the part that lets it maintain internal state between tokens so that it can plan longer responses? What part allows it to change 'rethink' its response when a bad dice roll derails a sentence? Like I've said before, once you peek behind the curtain, the magic vanishes.
One can quibble over how much or to what extent
You really can't. Such a thing is very obviously beyond the capabilities of these kinds of models. There just isn't any sufficient mechanism. People made similar claims about Joe Weizenbaum's Eliza, convinced that the machine truly understood them and empathized with their problems.
[...bunch of stuff...] using better grammar than I have ever been able to muster
If we're dramatically over-stating claims here, an n-gram of sufficient size can also generate convincing text. Of course, no one in their right mind would say that an n-gram model is creative or that it could reason! What do you think is the essential difference between the two? They generate text in very similar ways. Both produce text probabilistically, in constant time, without maintaining any internal state, one token at a time. Which part has the magic that makes one capable of reasoning and creativity and not the other? Be specific.
Re: (Score:2)
Squabbles like this are starting to erode my faith in future iterations of the Computer Sciences.
Next, they'll be constructing buildings under a coordinated logo/banner, trying to prove to the world how right they are by nature of their "success".
You're job is to fuck around with a fancy-ass graphing calculator to appease your boss, so that they decide to keep paying you - please don't pretend to be heralding a new form of life/consciousness/messiah, no one with any real sense will buy that hot trash.
Re: (Score:2)
It sounds like your experience with LLMs has been way different than mine. My experience is that they mostly get basic things wrong (with confidence) and lie about not knowing things.
Re: (Score:2)
I've experienced about the same. I wouldn't say it's mostly wrong, but it is certainly often wrong.
I created a prompt to collaborate with the model on a story set in a furniture store at an Amish community.
Within a few prompts it described an Amish woman's flowing auburn hair that extended below her shoulders.
Not something you would see out in public. So I went back to my initial prompt and included the way Amish women in this community dress.
After about 5 prompts it repeated it's earlier mistake and descri
Re: (Score:2)
Fascinating. So not even a basic understanding of things that must obviously have been in the training data. Incidentally, DDG gave me "Amish women don't display their hair for Biblically-based reasons" in the 2nd hit and without looking at the actual pages found.
Re: (Score:2)
I think we just have some people being dumb here because they are deeply in the thrall of wishful thinking. Hence they ignore all the crap and when an LLM outputs something smart by accident or because it was in the training data, they scream "Heureka!" and claim the machine is intelligent.
Re: (Score:2)
LLMs are modeled off of the way that the brain is thought to fundamentally behave, with probabilistic outcomes being influenced by the different potentials that exist between neural synapses.
When people such as yourself say "LLMs can not reason because the output is based on probabilities", you necessitate saying the exact same thing about humans.
Re: (Score:2)
If we make AI creative, and it can replicate other AIs. it might decide humans are just unnecessary for its existence... and then we have a problem. ;) So yeah, it should better not start being creative.
No they are very complex parrot (Score:2)
Re: (Score:2)
Indeed. But somehow some people have extreme delusions regarding what LLMs can do. No idea what their problem is, although religion may be dependent on a similar mechanism IMO.
Re: (Score:2)
Untrue. Unless this was a bad "ad hominem" attempt
A human being receives data both from other humans (parents, teachers, everyone else around them) as well as from nature itself. A computer can't trip and fall in mud, or get wet by rain, be cold, warm, be hit by pungent smells, have a tummy ache and so on. A human being does experience all of that, and a lot more.
Re: (Score:2)
I should have written this reply [slashdot.org] here.
Re: (Score:2)
My LLM thinks you suffer from datafixation, but since I couldn't find the word I it for a definition, see below.
Datafixation (noun):
- The erroneous belief or assertion that artificial intelligence or other data-driven systems are strictly bound or limited by the data they have been provided with, negating the potential for novel outputs, creativity, or the generation of unforeseen solutions through complex processing, pattern recognition, or other computational mechanisms.
Re: (Score:2)
*I* can be more creative with the data known to me, but an AI can't. If it somehow can create new data from the old, we should fear it. I don't fear it at all.
In music circles there is a lot of talk about AI, and what we found out is that yes - it can absolutely make music, but it's always similar to the data the program had been fed with. Which is absolutely normal. How to make AI think creative and not just replicate and remix something new from the data known to it is an actual problem, but also a proble
Re: (Score:2)
*I* can be more creative with the data known to me, but an AI can't. If it somehow can create new data from the old, we should fear it.
How does one go about discerning between something that is creative vs. something that is not? Is there an objective means of discrimination or is this merely expressing a subjective opinion?
Because it is capable of analysing mountains of data more efficiently than a normal human can, it is capable of and very useful for many things, but inventing new things and being truly creative is not one of those things.
Despite decades of intensive expert human effort AI figured out how to find lowest energy conformations of proteins while humans failed. Ditto for multiplying 4x4 matrices in fewer steps than previously known possible.
General capability for AI to come up with novel solutions has existed for decades using little more t
Re: (Score:2)
> These things cannot _reason_!
Maybe it doesn't need to. You are presuming that all intelligence must resemble human intelligence. That's merely an assumption that could be wrong.
There are indirect ways to "reason". For example, evolution didn't "reason" to design complex modes of life.
Re: (Score:2)
... You are presuming that all intelligence must resemble human intelligence. That's merely an assumption that could be wrong.
There are indirect ways to "reason". For example, evolution didn't "reason" to design complex modes of life.
I think that's a brilliant distinction you just made. I wonder if it might explain some of the strong disagreements between the experts who fear that LLM's are already developing autonomous intelligence, and the experts who laugh at the thought of that ever happening.
Re: (Score:2)
To find whether P = NP or not? Seriously? This is _mathematical_ reasoning and it is not even tied to the physical universe, just some very abstract computing models.
Re: (Score:2)
I asked ChatGPT 3.5 "If B implies C and C implies D and A implies B and A is not true, what can you say about D?" and it concluded D must be false. However I asked Bing (ChatGPT 4), and it correctly concluded you couldn't say anything about B or C or D, other than if B or C were true then D would be true.
I asked ChatGPT 3.5 "what is 30473*56782?" and it said "30,473 * 56,782 = 1,733,333,686." However I asked Bing (ChatGPT 4) the same thing and it said "That is a simple arithmetic question. The answer is 1
Re: (Score:2)
The full ChatGPT 3.5 response:
If B implies C, C implies D, and A implies B, and A is not true (i.e., A is false), we can use logical inference to determine the implication on D. Let's break it down step by step:
B implies C: This means that if B is true, then C must also be true.
C implies D: Similarly, if C is true, then D must also be true.
A implies B: If A is true, then B must be true. However, it's given that A is not true (i.e., A is false).
Since A is false and A implies B, we can conclude that B must al
Re: (Score:2)
This is hilarious! obviously there is only 1 answer for 30473*56782, yet another AI gave me 1,729,274,566, when the correct answer is 1,730,317,886.
Imagine calculating fuel consumption and airspeed for a flight and it's off by 10%. Sure hope you don't run out of fuel before you reach the airport, lol!
Re: (Score:2)
ChatGPT can reason. To a degree. You can totally ask it to explain something and it will. It is reasoning.
That is just a facsimile of reasoning. Actual reasoning involves inferences based on new (or previously unconnected) information. ChatGPT used to think cyanoacrylate was cured by acid. I told it "no, you've got that backwards, but please explain the chemistry again". It apologized and told me I was right, but said the same thing in a different way ("low pH" instead of "acid").
It's been fixed (I reported it; not sure if that helped) but the experience helped me see these things are dumb as a brick when their
Re: (Score:2)
Nice example! And it must have had enough actually correct chemistry in its training data to get that right. It did not. The thing is a lot dumber than what it was trained on.
Re: (Score:2)
By that "reasoning" a systematic catalog in a library, together with the books in that library is "can reason". Seriously?
No (Score:2)
I doubt we have words or concepts to solve the n/np problem. So how would the AI communicate it to us? It would need a new type of instructional ai capable of writing entire books in on go as an answer to a prompt.
Re: (Score:2)
Only if generative AI... (Score:3)
Nonsense (Score:5, Informative)
It took me an hour to convince the damn thing that binary tree mazes were perfect mazes, the single most obvious fact about binary tree mazes. Hell, it can't do simple arithmetic, it sure as hell isn't doing advanced mathematics.
Oh, the 74 page paper is almost entirely their chat log. The whole point of the thing seems to be to promote their nonsense term "Socratic reasoning" so that they can snag a few citations. (Socrates believed that there was no learning, only remembering. He 'proved' this to Meno by walking a slave, with no education, through some geometry problem by asking him leading questions.)
Don't waste your time.
Re:Nonsense (Score:4, Interesting)
Hell, it can't do simple arithmetic, it sure as hell isn't doing advanced mathematics.
That is non-sequitur. Being bad at one component of mathematics doesn't mean you can't be good at another. This has been shown here in Slashdot articles where AI screws up basic arithmetic (as you say), while also producing record breaking solutions to 4x4 matrix multiplication optimising the 50 year old record held by Strassen's algorithm. Or stories about how AI can't solve basic logic and yet can get perfect scores on IQ test (which is often largely a logic test).
Re:Nonsense (Score:5, Interesting)
FWIW the matrix multiplication claim is overstated in the abstract of the paper, and all of the reporting on it only parroted the abstract. The claim that it reduces 49 multiplications to 47 multiplications for 4x4 matrices sounds just about useful when you consider that multiplication is much more expensive than addition. But the detail that you have to read the full body of the paper to find out is that this reduction is only applicable over the field of two elements, and there multiplication isn't more expensive than addition. Some of the other results from the paper may have practical utility, but the headline one doesn't even have theoretical utility.
Re: (Score:2)
The claim that it reduces 49 multiplications to 47 multiplications for 4x4 matrices sounds just about useful when you consider that multiplication is much more expensive than addition.
Whether it is useful or not is irrelevant to the point. The point is that not being able to do arithmetic doesn't mean you can't use something to develop or optimise algorithms, which was objectively done regardless of how useless it may be.
Re: (Score:2)
We could have a battle of anecdotes, or we could recognize one very simple fact: LLMs are equivalent in computational power to lookup tables. That puts some hard limits on their capabilities.
It's easy to fool yourself into thinking more is happening that is actually happening. It's why I always recommend people take the time to learn about how these things function. Once you peek behind the curtain, the magic vanishes. Contrary to popular delusion, these aren't black boxes. We understand quite a bit a
Re: (Score:2)
or we could recognize one very simple fact: LLMs are equivalent in computational power to lookup tables.
What does this mean? Can you describe the equivalence in real terms?
It's easy to fool yourself into thinking more is happening that is actually happening.
It's easy to fool yourself into discounting anything you don't want to accept.
It's why I always recommend people take the time to learn about how these things function. Once you peek behind the curtain, the magic vanishes. Contrary to popular delusion, these aren't black boxes.
Knowing how something works doesn't mean you know what it's doing. While one might know how a digital computer works it doesn't mean they understand what an arbitrary piece of software running on that computer is doing.
The fact that people are training systems based on empirical experience rather than first principals, waste millions on dead ends and continue to
Re: (Score:2)
What does this mean? Can you describe the equivalence in real terms?
This is CS 101. Do a search for "classes of automata".
It's easy to fool yourself into discounting anything you don't want to accept.
It's a lot harder when you're actually informed.
emergent behavior means that
"Emergent behavior" doesn't mean "magic", you know.
Re: (Score:2)
What does this mean? Can you describe the equivalence in real terms?
This is CS 101. Do a search for "classes of automata".
Your assertion you explain what it even means let alone support it. If you don't feel like doing that you should expect to be ignored.
"Emergent behavior" doesn't mean "magic", you know.
What is the relevance? Who is talking about "magic"?
Re: (Score:2)
We could have a battle of anecdotes, or we could recognize one very simple fact: LLMs are equivalent in computational power to lookup tables.
And none of that precludes anything I've said. Sometimes the best way to generate an algorithm is to iterate analysis of a giant amount of information. Point is we have evidence of AIs developing optimised algorithms for problem solving. This isn't us battling anecdotes, this is me saying your anecdote is not relevant here as it is the wrong anecdote of capabilities for the application of a problem.
Re: (Score:2)
A better analogy is that animals don't need to be good at arithmetic to be good at foraging. However, foraging is not advanced mathematics.
Re: (Score:2)
It doesn't need to be impressive. A theoretical breakthrough is precisely what is being sought after here. The fact that you may not find the particular example impressive is not withstanding to the point that it was objectively made.... by a machine... which evidently can't do simple arithmetic.
Re: (Score:2)
Re: (Score:2)
It took me an hour to convince the damn thing that binary tree mazes were perfect mazes.
Your LLM service provider would like to thank you for donating your time and expertise to train their model and grow their business.
Re: (Score:2)
Ha! You're not wrong. In trying to solve it, I've become part of the problem.
Re: (Score:2)
Ha! You're not wrong. In trying to solve it, I've become part of the problem.
And that's about as human as you can get.
Re: (Score:2)
More wishful thinking... It's deeply disturbing, like seeing an adult convinced that a puppet is a real person because it moves and talks. This must be how Joe Weizenbaum felt about people's reactions to Eliza...
It can do simple arithmetic.
No, it can't. This is well-established. Also, remember that LLMs are equivalent in computational power to lookup tables. The limitations here are well-understood.
In a single pass it can only
A 'pass' here would be the output of a single token. See my other post for the basics. A whole response is as many 'passes' as there a
Re: (Score:2)
You sure love ignoring research and substituting ignorant insults instead, don't you?
A study on the capabilities of LLMs for you [arxiv.org].
User:6+7
ChatGPT: 6 + 7 equals 13.
User: 8/2
ChatGPT: 8 divided by 2 equals 4.
User: 4.3 modulo 2
ChatGPT: The modulo operation finds the remainder when one number is divided by another.
So, 4.3 modulo 2 would
Re: (Score:2)
It is well established that it can't do arithmetic, though if can look up answers that exist in its training set if they exist, and must be simple for their to be a reasonable chance that they do. No amount of looking up canned answers is "doing math".
If you actually read the paper you posted a link to their conclusion is essentially (with a long winded analysis) that it can't do math (unless making lots of errors counts). What they cite as evidence for any math ability is linguistic, not surprisingly --
Re: (Score:2)
This is really funny. You should try actually reading the paper you linked.
Re: (Score:2)
I regret that I posted on this page, because I can't downvote the posts where you were unnecessarily rude to people.
Re: (Score:2)
You get what you give. I treat everyone how they prefer to be treated, according to how they treat others. I put the same or better effort into my posts as the person to which I replied put into theirs. It's really that simple.
Re: (Score:2)
It can however do complex arithmetic when prompted to do things step-by-step, e.g. with many successive prompts asking only for the next step to be solved, rather than the whole problem.
Not all tokens need to generate a visible output. Cannot an LLM iterate internally (generating only discarded tokens to the output) until it finishes and then spitting out the final answer? I see it is not probable considering how they are trained but it looks possible.
Re: (Score:2)
Not all tokens need to generate a visible output. Cannot an LLM iterate internally (generating only discarded tokens to the output) until it finishes and then spitting out the final answer? I see it is not probable considering how they are trained but it looks possible.
While of course you can have a system that by convention intentionally hides reasoning steps in the presentation the same way EOS et el are signaled.. yet whatever tokens are generated as a result of reasoning have to be meaningful... it can't just spit out generic "I'm thinking" whitespace tokens and it can't not burn thru context in the process.
Re: (Score:2)
It can do simple arithmetic.
It can look up answers to simple arithmetic problems and repeat them if they appeared in its training set. If problem is complex enough (multiplying two 3 digit numbers seems to usually meet this criteria) it frequently fails as the odds that this particular problem ever appeared in its training set is low. This isn't "doing arithmetic" - the LLMs don't even have a model of what numbers are (though they can quote papers explaining numbers).
You are claiming elsewhere here "I literally train neural networks"
Re: (Score:2)
It can do simple arithmetic.
It can't even count. Ask it for a poem with six stanzas and watch it generate seven.
the single most important unsolved problem in comp (Score:2)
Say someone does come up with an answer. What changes?
Re:the single most important unsolved problem in c (Score:4, Interesting)
If it's a person who comes up with it, not much. Headlines, fame, prizes.
If it's an AI that beats people to it, I think it'll cause a lot of people to recontextualize their views on AI.
Re: (Score:2)
All will follow the AI. Get your knee pads ready. There is fixing to be a cock coddling like you have never seen.
Scientists discover the rubber duck method (Score:2)
Maybe this will be the killer application for large language models
Can AI generate itself? (Score:2)
If a given AI could generate another AI with the same Kolmogorov complexity then would P = NP? If not then there must exist a hierarchy of complexity classes and makes me then think P must not equal NP.
Re: (Score:2)
Does another copy (instance) of the AI itself count?
Any useful AI must be able to process some sensory input. That is another information added to the AI from the outside world. It will increase its complexity right there.
Re:Can AI generate itself? (Score:4, Informative)
Re: (Score:2)
A 3-digit UID is a badge of honor...a little for the holder but maybe more for the site that has new UIDs at 7 or 8 digits now. This Thursday it will be 26 years.
I had a 4-digit UID, but took some time off and forgot my pw in the downtime. I created a new account only 206k-ish people later.
Re: (Score:2)
https://www.youtube.com/watch?... [youtube.com]
The paper is about AIÂ*helping* to solve prob (Score:5, Insightful)
No, but (Score:2)
It may be a really useful tool that allows researchers to easily become aware of ideas being worked on by others
There are way too many papers to read, and it might turn out than an insight gained in another field may be helpful
Nomad (Score:2)
Nomad, can you tell me how you're going to solve this problem without actually solving it?
Blows up.
Short answer: no. (Score:2)
Longer answer: hell no!
Easy answer , no (Score:2)
Well, no, it can't. But moreover, computer "science" isn't a science, it's a trade, and solving the p=np problem would have no consequential effect on anyone, it's important in the same sense that an autistic person knocks a door 2 and becomes obsessed because they never did the third.
Generative AI could solve it if (Score:2)
it finds and summarizes an article written by a human that explains the solution.
Seriously, generative AI doesn't find solutions, in the sense of coming up with answers that didn't previously exist. It finds solutions in the sense of locating them on the internet, and summarizing them. And even then, it often gets it wrong, if it finds source articles that are wrong.
GPT4 only regurgitates what we know (Score:2)
Re: (Score:2)
It cannot prove or know something outside of what we told it.
Can you?
Re: (Score:2)
Yes. Humans have the ability to infer. "AI" does not.
It's literally called the inference problem, one of the biggest roadblocks in AI that we have no idea how to overcome, but which you can witness even a dog perform in some manner.
Sonny from iRobot being pedantic about the use of the word "you" or "I" to not encapsulate humans in general is just being petty.
Because pretty much every human can infer better than any AI.
Without doubt (Score:2)