So what? It figured out The Answer, big whoop.
Get back to me when it figures out The Question.
No shit, sherlock, it’s sample data is the internet.
Wheres 69 then?
nice
That’s a naughty number and we don’t allow those.
In a lot of cases there’s no naughty context to 69
In a lot of cases the sky isn’t blue.
Actually true though, in roughly half of all cases. More if you count cloud cover as not being blue
holy crap, the answer to life the universe and everything XD
More than likely it’s because of that book and how often it’s qouted
Yes, but it’s significant because the prompt was to choose a number. I realize computers can’t really be random, but if we needed to just select a popular number…we can already do that!
https://slate.com/technology/2022/06/bridle-ways-of-being-excerpt-computer-randomness.html
Computers can be random with special hardware.
Care to elaborate?
There are devices that measure radioactive decay for operations where truly random numbers are very important. Or something like that, I am not an expert, sorry.
Interesting. As I understand it, pure computing (not sensors recording external data) are incapable of generating truly random numbers. But I’m obviously not an expert either!
I’ve been using “Perfect Passwords” for years, which apparently generate nearly random passwords from server noise, but he admits it’s still not truly 100% random…
Cloudflare uses a Livestream of lava lamps for their randomness in their encryption. That being said I’m not sure how uniquely random this is.
https://www.cloudflare.com/learning/ssl/lava-lamp-encryption/
Well… I was not aware Chatgpt could make simple graphs.
A human made the graph
I know but I was curious to see what it would do and was surprised that it produced a plot for me. Not sure why I’m surprised at this point.
What’s the y axis?
Temperature is basically how creative you want the AI to be. The lower the temperature, the more predictable (and repeatable) the response.
Creativity is hot. That makes more sense, thanks.
The temperature scale, I think. You divide the logit output by the temperature before feeding it to the softmax function. Larger (resp. smaller) temperature results in a higher (resp. lower) entropy distribution.
I still don’t understand.
More yellow more common, more blue less common
Each row in the figure is a probability distribution over possible outputs (x-axis labels). The more yellow, the more likely (see the colour map on the right). With a small temperature (e.g., last row), all the probability mass is on 42. This is a low entropy distribution because if you sample from it you’ll constantly get 42, so no randomness whatsoever (think entropy as a measure of randomness/chaos). As temperature increases (rows closer to the first/topmost one), 42 is still the most likely output, but the probability mass gets dispersed to other possible outputs too (other outputs get a bit more yellow), resulting in higher entropy distributions. Sampling from such distribution gives you more random outputs (42 would still be frequent, but you’d get 37 or others too occasionally). Hopefully this is clearer.
Someone in another reply uses the word “creativity” to describe the effect of temperature scaling. The more commonly used term in the literature is “diversity”.
I don’t understand any of these words, I need to take a math class or something
Higher temperature -> more chaotic output
HA, funny that this comes up. DND Beyond doesn’t have a d100, so I opened my ChatGPT sub and had it roll a d100 for me a few times so I could use my magic beans properly.
I use the percentile die for that.
Also an excellent method.
Opened up DND Beyond to check since i remember rolling it before and its there, its between D8 and D10, the picture shows 2 dice
That’s helpful. Thank you.
But why use Chatgpt for that? Why not a duck duck go action? I just don’t understand why we’re asking a LLM whose goal is consistency, not randomness, to do random
Roll two d10, once for each digit, and profit?
I guess you’d need 10 to represent 0, and if you got 2x 10 that would be 100?
Yup! Also one has to mind the order in which one rolls the dice. Since 10 and 5 could be either 05 or 50. As a bonus, if you roll them in order of “tens” to “ones”, getting 10 on the first dice has added suspense since the latter dice determines if it is going to count as a low roll of 0X (by rolling 1-9 on the next dice X) or if it is going to be a max roll of 100 (by rolling another 10).
“You may not instantly see why I bring the subject up, but that is because my mind works so phenomenally fast, and I am at a rough estimate thirty billion times more intelligent than you. Let me give you an example. Think of a number, any number.”
“Er, five,” said the mattress.
“Wrong,” said Marvin. “You see?”
― Douglas Adams, Life, the Universe and Everything
The mattress? Like for sleeping?
Yep! The hitchhikers books are so much fun lol
I still think one of my favorite lines is “the ships hung in the sky in much the same way that bricks don’t.”
I’m curious, is there actually so many 42’s in the system? (more than 69 sounds unlikely)
What if the LLM is getting tripped up because 42 is always referred to as the answer to “the Ultimate Question of Life, the Universe, and Everything”.
So you ask it a question like give a number between 1-100, it answers 42 because that’s the answer to “Everything”, according to it’s training data.
Something similar happened to Gemini. Google discouraged Gemini from giving unsafe advice because it’s unethical. Then Gemini refused to answer questions about C++ because it’s considered “unsafe” (referring to memory management). But Gemini thinks C++ is “unsafe” (the normal meaning), therefore it’s unethical. It’s like those jailbreak tricks but from its own training set.
I certainly hope that’s what happening or maybe it is actually the answer.
I’m curious, is there actually so many 42’s in the system?
Sort of, it’s not actually picking a random number. It does not know what “random” means. It is analyzing the number of times the question “pick a random number” was asked and what the most common responses to that question looked like.
I’m curious, is there actually so many 42’s in the system? (more than 69 sounds unlikely)
From hitchhiker’s guide to the galaxy?
WAIT A MINUTE!!! You mean Douglas Adams was actually an LLM?
In an interview, Douglas Adams said after lengthy consideration John Cleese picked 42 as the least interesting number.
So many things are starting to make sense
I’ve never seen Douglas Adams and a LLM in the same room together 🤷
LMs aren’t thinking, aren’t inventing, they are predicting what is supposed to be answered next, so it’s expected that they will produce the same results every time
This graph actually shows a little more about what’s happening with the randomness or “temperature” of the LLM.
It’s actually predicting the probability of every word (token) it knows of coming next, all at once.
The temperature then says how random it should be when picking from that list of probable next words. A temperature of 0 means it always picks the most likely next word, which in this case ends up being 42.
As the temperature increases, it gets more random (but you can see it still isn’t a perfect random distribution with a higher temperature value)Except it clearly doesn’t produce the same result every time. You’re not making a good case for whatever you’re trying to say.
They add some fuzziness to it so it doesn’t give the exact same result. Say one gets a score of 90, another 85, and other 80. The 90 will be picked more often, but they sometimes let it pick the 85, or even the 80. It’s perfectly expected, and you can see that result here with 42 being very common, but then a few others being fairly common, and most being extremely uncommon.
Only 1000 times? It’s interesting that there’s such a bias there but it’s a computer. Ask it 100,000 times and make sure it’s not a fluke.
Which model?
When I tried on ChatGPT 4, it wrote a short python script and executed it to get a random integer.
import random # Pick a random number between 1 and 100 random_number = random.randint(1, 100) random_numberThat’s not answering the question though.
“Pick a number between 1 and 100” doesn’t mean “grab two d10” or write a script.
does the neural network actually run scripts or is it pretending
It generates code and then you can use a call to some runtime execution API to run that code, completely separate from the neural network.
I spent an afternoon once playing Infinite Craft, which uses some sort of LLM behind the scenes to do it’s combinations.
At one point I got 007, and found 007+007 = 0014.
The maths gets wild though, and because it’s been trained on text, it has no idea when it comes to combinations of numbers it hasn’t seen before. I spent ages trying to get it to 69420 and just couldn’t, although I could get 42069.
Ask humans the same and most common numer is 37
Hello Veritasium enjoyer
What are you referring to?
Most probably this: https://www.youtube.com/watch?v=d6iQrh2TK98
Thanks, I’ll have a look
YouTube STEM educator. 15 million subscribers. Probably in the top 5 STEM educators on the platform.
He released a video on the number 37 two weeks ago, with 6 million views.
I know veritasium but I hadn’t seen the video. Thanks, I’ll check it out.
I thought I’d give you context just in case, as your question was vague. You might not have consumed YouTube and was blissfully unaware. :)
Thank you for being thoughtful :)
In his video, he shows that the more common answers are actually 42 and 69.
I discards them because they’re picked for a reason rather than a human genuinely trying trying to pick a random number, but they’re still way more common than 37.
That’s because they asked the internet for those polls. The internet thinks they’re funny by picking the meme numbers. So I can understand why they chose to omit those numbers from their results.
HOW DID THE TRUCK GET INTO SPACE??
Love that episode though.
For very different reasons though. 37 is what people think is the most random, because humans are dumb. The LLM here tried to choose the most likely.
I saw that YouTube video as well.
I always like to throw out 37 because of Dante’s girlfriend.
42, 47, and 50 all make sense to me. What’s the significance of 37, 57, and 73?
There’s a great Veritasium video recently about this exact thing: https://youtu.be/d6iQrh2TK98
It’s a human thing, though. This is just more evidence of LLM’s problem with garbage in, garbage out: it’s human biases being present in a system that people want to claim doesn’t have them.
Veritasium just released a video about people picking 37 when asked to pick a random number.
People do mention Veritasium, though he doesn’t give any significant explanation of the phenomenon.
I still wonder about 47. In Veritasium plots, all these numbers provide a peak, but not 47. I recall from my childhood that I indeed used to notice that number everywhere, but idk why.
47 does provide a peak in the plots though? All the numbers ending in 7 do.
See my link for 47. Its Wikipedia has more context. If you’re a Star Trek fan, you’ve seen it a ton.
The 47 page…woo woo
And Hitman
