Daniël de Zeeuw: A combination of genuine concerns and moral panics over AI (often fueled by the tech moguls themselves) has reached a new high in recent years, particularly with the introduction of ChatGPT by OpenAI.

Doomsaying can become a self-fulfilling prophecy. Produced by ElevenLabs and News Over Audio (NOA) using AI narration.

Let me start by saying a few things that seem obvious,” Geoffrey Hinton, “Godfather” of deep learning, and one of the most celebrated scientists of our time, told a leading AI conference in Toronto in 2016.

The introduction to the book Metaverse Evolution Theory by Japanese VTuber and writer Virtual Girl Nem has now been published in English. The 5,000-word introduction is available at her official Medium page, and aimed at generating discussion about her book in the English-speaking community.

The term Luddite is usually used as an insult. It suggests someone who is backward-looking, averse to progress, afraid of new technology, and frankly, not that bright. But Brian Merchant claims that that is not who the Luddites were at all.

It is estimated that one language dies every 14 days, with it a world of concepts and ideas that could only be expressed in its words. Most of the languages at risk of extinction belong to Indigenous communities.

The video site isn’t just a platform. It’s infrastructure. Produced by ElevenLabs and NOA, News Over Audio, using AI narration.

Do you like Star Wars lore? Exactly how much do you like Star Wars lore? I ask because A Very Brief Analysis: The Phantom Menace is 12 hours long and is chock full of Star Wars lore. "This is not a defence of Episode 1, nor is it an attack. The idea is a detailed, fair, and informed analysis. We'll go over the entire movie, giving credit and blame as needed."

It's time to talk about the Ludding times. That's how they talked about it, those who were there, when readying themselves for an insurrection a few years later.

This video explores the journey of AI language models, from their modest beginnings through the development of OpenAI's GPT models. Our journey takes us through the key moments in generative neural network research involved in next word prediction. We delve into the early experiments with tiny language models in the 1980s, highlighting significant contributions by researchers like Jordan, who introduced Recurrent Neural Networks, and Elman, whose work on learning word boundaries revolutionized our understanding of language processing. It leaves us with a question: what is thought? Is simulated thought, thought? Featuring Noam Chomsky Douglas Hofstadter Michael I. Jordan Jeffrey Elman Geoffrey Hinton Ilya Sutskever Andrej Karpathy Yann LeCun and more. (Sam altman)

My script, references & visualizations here: https://docs.google.com/document/d/1s7FNPoKPW9y3EhvzNgexJaEG2pP4Fx_rmI4askoKZPA

consider joining my channel as a YouTube member: https://www.youtube.com/channel/UCotwjyJnb-4KW7bmsOoLfkg/join

This is the last video in the series "The Pattern Machine" you can watch it all here: https://www.youtube.com/playlist?list=PLbg3ZX2pWlgKV8K6bFJr5dhM7oOClExUJ

00:00 - Introduction 00:32 - hofstader's thoughts on chatGPT 01:00 - recap of supervised learning 01:55 - first paper on sequential learning 02:55 - first use of state units (RNN) 04:33 - first observation of word boundary detection 05:30 - first observation of word clustering 07:16 - first "large" language model Hinton/Sutskever 10:10 - sentiment neuron (Ilya | OpenAI) 12:30 - transformer explaination 15:50 - GPT-1 17:00 - GPT-2 17:55 - GPT-3 18:20 - In-context learning 19:40 - ChatGPT 21:10 - tool use 23:25 - philosophical question: what is thought?

 THE BAD NEWS is waiting for us on the first page of the first chapter of Octavia E. Butler’s 1998 science fiction novel Parable of the Talents.

If you plug “AI” or “artificial intelligence” into Google Image Search, here’s what you get. Electric brains firing blue with neurons. Translucent robots and robot heads, code dribbling down their foreheads and noses.

Whatever future comes, one thing is fairly certain, excepting total American collapse—the goods will continue to flow. By human, by vehicle, by automaton, or by some combination of the three,stuff will be distributed, sold, and speculated upon across the land.

“Men have to toughen up,” Jordan B. Peterson writes in 12 Rules For Life: An Antidote to Chaos, “Men demand it, and women want it.” So, the first rule is, “Stand up straight with your shoulders back” and don’t forget to “clean your room.

Of all the buzzy 21st-century tech phrases, “machine learning” threatens to be the most important. Programming computers is slow, but we’re nearing the point where humans give the bots parameters and let them teach themselves.

But even after I overcame my instinct for detachment, I remained wary of the movement’s language, which was a language of binaries: women and men.

It has long been claimed that pornography drives technology. This claim is over-stated, no doubt, but what’s clear is that pornography very quickly and cleverly takes advantage of new communication technologies.

Undiagnosed by the mainstream media and much of the academic community, a major intellectual renewal is underway across the left. It is energetic and tech-savvy, building platforms such as Novara Media.

Warning: Contains strobing effects. A short animated documentary by Eryk Salvaggio on how AI image generation works, and the entanglement of composite photography,…

This is a 1 hour general-audience introduction to Large Language Models: the core technical component behind systems like ChatGPT, Claude, and Bard. What they are, where they are headed, comparisons and analogies to present-day operating systems, and some of the security-related challenges of this new computing paradigm. As of November 2023 (this field moves fast!).

Context: This video is based on the slides of a talk I gave recently at the AI Security Summit. The talk was not recorded but a lot of people came to me after and told me they liked it. Seeing as I had already put in one long weekend of work to make the slides, I decided to just tune them a bit, record this round 2 of the talk and upload it here on YouTube. Pardon the random background, that's my hotel room during the thanksgiving break.

• Slides as PDF: https://drive.google.com/file/d/1pxx_ZI7O-Nwl7ZLNk5hI3WzAsTLwvNU7/view?usp=share_link (42MB)
• Slides. as Keynote: https://drive.google.com/file/d/1FPUpFMiCkMRKPFjhi9MAhby68MHVqe8u/view?usp=share_link (140MB)

Few things I wish I said (I'll add items here as they come up): - The dreams and hallucinations do not get fixed with finetuning. Finetuning just "directs" the dreams into "helpful assistant dreams". Always be careful with what LLMs tell you, especially if they are telling you something from memory alone. That said, similar to a human, if the LLM used browsing or retrieval and the answer made its way into the "working memory" of its context window, you can trust the LLM a bit more to process that information into the final answer. But TLDR right now, do not trust what LLMs say or do. For example, in the tools section, I'd always recommend double-checking the math/code the LLM did. - How does the LLM use a tool like the browser? It emits special words, e.g. |BROWSER|. When the code "above" that is inferencing the LLM detects these words it captures the output that follows, sends it off to a tool, comes back with the result and continues the generation. How does the LLM know to emit these special words? Finetuning datasets teach it how and when to browse, by example. And/or the instructions for tool use can also be automatically placed in the context window (in the “system message”). - You might also enjoy my 2015 blog post "Unreasonable Effectiveness of Recurrent Neural Networks". The way we obtain base models today is pretty much identical on a high level, except the RNN is swapped for a Transformer. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ - What is in the run.c file? A bit more full-featured 1000-line version hre: https://github.com/karpathy/llama2.c/blob/master/run.c

Chapters: Part 1: LLMs 00:00:00 Intro: Large Language Model (LLM) talk 00:00:20 LLM Inference 00:04:17 LLM Training 00:08:58 LLM dreams 00:11:22 How do they work? 00:14:14 Finetuning into an Assistant 00:17:52 Summary so far 00:21:05 Appendix: Comparisons, Labeling docs, RLHF, Synthetic data, Leaderboard Part 2: Future of LLMs 00:25:43 LLM Scaling Laws 00:27:43 Tool Use (Browser, Calculator, Interpreter, DALL-E) 00:33:32 Multimodality (Vision, Audio) 00:35:00 Thinking, System 1/2 00:38:02 Self-improvement, LLM AlphaGo 00:40:45 LLM Customization, GPTs store 00:42:15 LLM OS Part 3: LLM Security 00:45:43 LLM Security Intro 00:46:14 Jailbreaks 00:51:30 Prompt Injection 00:56:23 Data poisoning 00:58:37 LLM Security conclusions End 00:59:23 Outro