Neural Networks – Intuitively and Exhaustively Explained

How can a person make up for seven decades of misrepresentation and willful distortion in the time allotted to a sound bite? Recently, an Australian-Palestinian friend of mine was invited to appear on Australia’s national television network to discuss the situation in and around Gaza.

Daniël de Zeeuw: A combination of genuine concerns and moral panics over AI (often fueled by the tech moguls themselves) has reached a new high in recent years, particularly with the introduction of ChatGPT by OpenAI.

This video explores the journey of AI language models, from their modest beginnings through the development of OpenAI's GPT models. Our journey takes us through the key moments in generative neural network research involved in next word prediction. We delve into the early experiments with tiny language models in the 1980s, highlighting significant contributions by researchers like Jordan, who introduced Recurrent Neural Networks, and Elman, whose work on learning word boundaries revolutionized our understanding of language processing. It leaves us with a question: what is thought? Is simulated thought, thought? Featuring Noam Chomsky Douglas Hofstadter Michael I. Jordan Jeffrey Elman Geoffrey Hinton Ilya Sutskever Andrej Karpathy Yann LeCun and more. (Sam altman)

My script, references & visualizations here: https://docs.google.com/document/d/1s7FNPoKPW9y3EhvzNgexJaEG2pP4Fx_rmI4askoKZPA

consider joining my channel as a YouTube member: https://www.youtube.com/channel/UCotwjyJnb-4KW7bmsOoLfkg/join

This is the last video in the series "The Pattern Machine" you can watch it all here: https://www.youtube.com/playlist?list=PLbg3ZX2pWlgKV8K6bFJr5dhM7oOClExUJ

00:00 - Introduction 00:32 - hofstader's thoughts on chatGPT 01:00 - recap of supervised learning 01:55 - first paper on sequential learning 02:55 - first use of state units (RNN) 04:33 - first observation of word boundary detection 05:30 - first observation of word clustering 07:16 - first "large" language model Hinton/Sutskever 10:10 - sentiment neuron (Ilya | OpenAI) 12:30 - transformer explaination 15:50 - GPT-1 17:00 - GPT-2 17:55 - GPT-3 18:20 - In-context learning 19:40 - ChatGPT 21:10 - tool use 23:25 - philosophical question: what is thought?

This is a 1 hour general-audience introduction to Large Language Models: the core technical component behind systems like ChatGPT, Claude, and Bard. What they are, where they are headed, comparisons and analogies to present-day operating systems, and some of the security-related challenges of this new computing paradigm. As of November 2023 (this field moves fast!).

Context: This video is based on the slides of a talk I gave recently at the AI Security Summit. The talk was not recorded but a lot of people came to me after and told me they liked it. Seeing as I had already put in one long weekend of work to make the slides, I decided to just tune them a bit, record this round 2 of the talk and upload it here on YouTube. Pardon the random background, that's my hotel room during the thanksgiving break.

• Slides as PDF: https://drive.google.com/file/d/1pxx_ZI7O-Nwl7ZLNk5hI3WzAsTLwvNU7/view?usp=share_link (42MB)
• Slides. as Keynote: https://drive.google.com/file/d/1FPUpFMiCkMRKPFjhi9MAhby68MHVqe8u/view?usp=share_link (140MB)

Few things I wish I said (I'll add items here as they come up): - The dreams and hallucinations do not get fixed with finetuning. Finetuning just "directs" the dreams into "helpful assistant dreams". Always be careful with what LLMs tell you, especially if they are telling you something from memory alone. That said, similar to a human, if the LLM used browsing or retrieval and the answer made its way into the "working memory" of its context window, you can trust the LLM a bit more to process that information into the final answer. But TLDR right now, do not trust what LLMs say or do. For example, in the tools section, I'd always recommend double-checking the math/code the LLM did. - How does the LLM use a tool like the browser? It emits special words, e.g. |BROWSER|. When the code "above" that is inferencing the LLM detects these words it captures the output that follows, sends it off to a tool, comes back with the result and continues the generation. How does the LLM know to emit these special words? Finetuning datasets teach it how and when to browse, by example. And/or the instructions for tool use can also be automatically placed in the context window (in the “system message”). - You might also enjoy my 2015 blog post "Unreasonable Effectiveness of Recurrent Neural Networks". The way we obtain base models today is pretty much identical on a high level, except the RNN is swapped for a Transformer. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ - What is in the run.c file? A bit more full-featured 1000-line version hre: https://github.com/karpathy/llama2.c/blob/master/run.c

Chapters: Part 1: LLMs 00:00:00 Intro: Large Language Model (LLM) talk 00:00:20 LLM Inference 00:04:17 LLM Training 00:08:58 LLM dreams 00:11:22 How do they work? 00:14:14 Finetuning into an Assistant 00:17:52 Summary so far 00:21:05 Appendix: Comparisons, Labeling docs, RLHF, Synthetic data, Leaderboard Part 2: Future of LLMs 00:25:43 LLM Scaling Laws 00:27:43 Tool Use (Browser, Calculator, Interpreter, DALL-E) 00:33:32 Multimodality (Vision, Audio) 00:35:00 Thinking, System 1/2 00:38:02 Self-improvement, LLM AlphaGo 00:40:45 LLM Customization, GPTs store 00:42:15 LLM OS Part 3: LLM Security 00:45:43 LLM Security Intro 00:46:14 Jailbreaks 00:51:30 Prompt Injection 00:56:23 Data poisoning 00:58:37 LLM Security conclusions End 00:59:23 Outro

The bartender pouring free drinks glances around with nothing to do as the raucous room full of startup founders and software developers gravitates toward its preferred drug: networking.

A video about neural networks, function approximation, machine learning, and mathematical building blocks. Dennis Nedry did nothing wrong. This is a submission for #SoME3

My Links Patreon: https://www.patreon.com/emergentgarden Discord: https://discord.gg/ZsrAAByEnr

Links and Content: On Mathematical Maturity, Thomas Garrity: https://www.youtube.com/watch?v=zHU1xH6Ogs4 Earth Rotation Loop: https://www.youtube.com/watch?v=aiQdLP2mBJE Modeling Shell Surfaces: https://www.geogebra.org/m/xtv7zpn5 Fourier Features Paper: https://arxiv.org/abs/2006.10739 Code for mandelbrot/image approximations: https://github.com/MaxRobinsonTheGreat/mandelbrotnn Code for line/surface approximations: https://github.com/MaxRobinsonTheGreat/ManimApproximations

Music: https://youtube.com/@acolyte-compositions

Timestamps (0:00) Functions Describe the World (3:15) Neural Architecture (5:35) Higher Dimensions (11:55) Taylor Series (15:20) Fourier Series (21:25) The Real World (24:32) An Open Challenge

An artificial intelligence trained to mimic the logic circuits of an ordinary computer can run code within itself, potentially speeding up certain calculations An artificial intelligence that mimics the operation of a standard computer within its neural network could speed up certain calculations.

An artificial intelligence trained to mimic the logic circuits of an ordinary computer can run code within itself, potentially speeding up certain calculations An artificial intelligence that mimics the operation of a standard computer within its neural network could speed up certain calculations.

In 2014 machine learning researcher Ian Goodfellow introduced the idea of generative adversarial networks or GANs.

The TikTok Cultures Research Network held its third virtual event TikTok Methodologies on 19 July 2021 hosted by founding members Associate Professor Crystal Abidin, Professor Patrik Wikström, and Dr D. Bondy Valdovinos Kaye.

The field of digital media research is rapidly expanding to include innovative and exciting TikTok research. As one of the most widely used digital short video platforms around the world, TikTok is a key social arena to study global youth culture, creativity, professionalization, and activism. This event was a critical forum for discussion on TikTok research frameworks and methodologies from emerging voices in the field.

Traditionally we think of images as relatively individualized or delimited phenomena that, in one way or the other, appear to the human mind and apparatus of perception. Currently, however, we are witnessing an intensification of what we might call thenetworkedness of the image along with a proliferation of machine imagery that operates independently of human perception and cognition. These “operative images” are, in Harun Farocki’s oft-cited formulation, “images without a social goal, not for edification, not for reflection”—they “do not represent an object, but rather are part of an operation.” In fact, one might claim that one of the characteristics of the contemporary image-space is its increasing integration of operational images and machine vision.

Relax with these peaceful visuals from Studio Ghibli! Stream your favorite Studio Ghibli films now on HBO Max!

Subscribe: http://bit.ly/HBOMaxYouTube

Be the first to know more: HBO Max: https://hbom.ax/YT Like HBO Max on Facebook: http://bit.ly/HBOMaxFacebook Follow HBO Max on Twitter: http://bit.ly/HBOMaxTwitter Follow HBO Max on Instagram: http://bit.ly/HBOMaxInstagram

About HBO Max: HBO Max is WarnerMedia’s direct-to-consumer offering with 10,000 hours of curated premium content. HBO Max offers powerhouse programming for everyone in the home, bringing together HBO, a robust slate of new original series, key third-party licensed programs and movies, and fan favorites from WarnerMedia’s rich library including Warner Bros., New Line, DC, CNN, TNT, TBS, truTV, Turner Classic Movies, Cartoon Network, Adult Swim, Crunchyroll, Rooster Teeth, Looney Tunes and more. #HBOMax #WarnerMedia

So this neural network is called Jukebox, by OpenAI, which basically generates songs that don't exist. Also bonus AI upscaled gifs of Rick Astley, which definitely doesn't make him look like Satan spawn.

Never Gonna Give You Up (AI Versions): https://jukebox.openai.com/?song=787729588 https://jukebox.openai.com/?song=787732303 https://jukebox.openai.com/?song=787735267 https://jukebox.openai.com/?song=787733749 https://jukebox.openai.com/?song=787730953

[Blog] https://openai.com/blog/jukebox/ [Sample Explorer] https://jukebox.openai.com/ [Source Code] https://github.com/openai/jukebox/ (You can generate your own songs, but you need a pretty beefy computer, some programming knowledge, and a lot of time and patience) [AI Upscaler] https://topazlabs.com/gigapixel-ai/ (There's a 30-day trial if you don't feel like paying for it)

Wow, this is the most views I've gotten on a video, like, ever. It even surpasses videos on my other channel.

[PLEASE READ THE CHANNEL DESCRIPTION]

Science is a network, each paper linking those that came before with those that followed. In an exclusive analysis, researchers have delved into Nature's part of that network. We explore their results, taking you on a tour of 150 years of interconnected, interdisciplinary research, as represented by Nature's publication record.

Explore the network yourself: https://www.nature.com/articles/d41586-019-03165-4 Read more: https://www.nature.com/collections/eidahgdici/

Donna Haraway famously pronounced, “the boundary between science fiction and social reality is an optical illusion” in her influential A Cyborg Manifesto.

This talk is about some pressing issues with AI that don't usually make the headlines, and why tackling those issues means developing an antifascist AI. When i talk about AI i'm talking about machine learning and about artificial neural networks, also known as deep learning1.

Some say in ice. The end of the world may come slowly, but it´s inevitable. Our sun, exhausting the hydrogen-fuel of the core, will successively burn the outer layers and doing so becoming hotter and expanding in size.

I logged into "Second Life" in the year 2018 A.D. It still exists, sort of. Residents and businesses began fleeing for more popular social networks long ago. Vast acres of land are abandoned or sparsely populated by the few remaining diehard users.

Memes and social networks have become weaponized, while many governments seem ill-equipped to understand the new reality of information warfare. How will we fight state-sponsored disinformation and propaganda in the future?