Large language models like those offered by OpenAI and Google famously require vast troves of training data to work. The latest versions of these models have already scoured much of the existing internet which has led some to fear there may not be enough new data left to train future iterations.

This video explores the journey of AI language models, from their modest beginnings through the development of OpenAI's GPT models. Our journey takes us through the key moments in generative neural network research involved in next word prediction. We delve into the early experiments with tiny language models in the 1980s, highlighting significant contributions by researchers like Jordan, who introduced Recurrent Neural Networks, and Elman, whose work on learning word boundaries revolutionized our understanding of language processing. It leaves us with a question: what is thought? Is simulated thought, thought? Featuring Noam Chomsky Douglas Hofstadter Michael I. Jordan Jeffrey Elman Geoffrey Hinton Ilya Sutskever Andrej Karpathy Yann LeCun and more. (Sam altman)

My script, references & visualizations here: https://docs.google.com/document/d/1s7FNPoKPW9y3EhvzNgexJaEG2pP4Fx_rmI4askoKZPA

consider joining my channel as a YouTube member: https://www.youtube.com/channel/UCotwjyJnb-4KW7bmsOoLfkg/join

This is the last video in the series "The Pattern Machine" you can watch it all here: https://www.youtube.com/playlist?list=PLbg3ZX2pWlgKV8K6bFJr5dhM7oOClExUJ

00:00 - Introduction 00:32 - hofstader's thoughts on chatGPT 01:00 - recap of supervised learning 01:55 - first paper on sequential learning 02:55 - first use of state units (RNN) 04:33 - first observation of word boundary detection 05:30 - first observation of word clustering 07:16 - first "large" language model Hinton/Sutskever 10:10 - sentiment neuron (Ilya | OpenAI) 12:30 - transformer explaination 15:50 - GPT-1 17:00 - GPT-2 17:55 - GPT-3 18:20 - In-context learning 19:40 - ChatGPT 21:10 - tool use 23:25 - philosophical question: what is thought?

If you plug “AI” or “artificial intelligence” into Google Image Search, here’s what you get. Electric brains firing blue with neurons. Translucent robots and robot heads, code dribbling down their foreheads and noses.

This is a 1 hour general-audience introduction to Large Language Models: the core technical component behind systems like ChatGPT, Claude, and Bard. What they are, where they are headed, comparisons and analogies to present-day operating systems, and some of the security-related challenges of this new computing paradigm. As of November 2023 (this field moves fast!).

Context: This video is based on the slides of a talk I gave recently at the AI Security Summit. The talk was not recorded but a lot of people came to me after and told me they liked it. Seeing as I had already put in one long weekend of work to make the slides, I decided to just tune them a bit, record this round 2 of the talk and upload it here on YouTube. Pardon the random background, that's my hotel room during the thanksgiving break.

• Slides as PDF: https://drive.google.com/file/d/1pxx_ZI7O-Nwl7ZLNk5hI3WzAsTLwvNU7/view?usp=share_link (42MB)
• Slides. as Keynote: https://drive.google.com/file/d/1FPUpFMiCkMRKPFjhi9MAhby68MHVqe8u/view?usp=share_link (140MB)

Few things I wish I said (I'll add items here as they come up): - The dreams and hallucinations do not get fixed with finetuning. Finetuning just "directs" the dreams into "helpful assistant dreams". Always be careful with what LLMs tell you, especially if they are telling you something from memory alone. That said, similar to a human, if the LLM used browsing or retrieval and the answer made its way into the "working memory" of its context window, you can trust the LLM a bit more to process that information into the final answer. But TLDR right now, do not trust what LLMs say or do. For example, in the tools section, I'd always recommend double-checking the math/code the LLM did. - How does the LLM use a tool like the browser? It emits special words, e.g. |BROWSER|. When the code "above" that is inferencing the LLM detects these words it captures the output that follows, sends it off to a tool, comes back with the result and continues the generation. How does the LLM know to emit these special words? Finetuning datasets teach it how and when to browse, by example. And/or the instructions for tool use can also be automatically placed in the context window (in the “system message”). - You might also enjoy my 2015 blog post "Unreasonable Effectiveness of Recurrent Neural Networks". The way we obtain base models today is pretty much identical on a high level, except the RNN is swapped for a Transformer. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ - What is in the run.c file? A bit more full-featured 1000-line version hre: https://github.com/karpathy/llama2.c/blob/master/run.c

Chapters: Part 1: LLMs 00:00:00 Intro: Large Language Model (LLM) talk 00:00:20 LLM Inference 00:04:17 LLM Training 00:08:58 LLM dreams 00:11:22 How do they work? 00:14:14 Finetuning into an Assistant 00:17:52 Summary so far 00:21:05 Appendix: Comparisons, Labeling docs, RLHF, Synthetic data, Leaderboard Part 2: Future of LLMs 00:25:43 LLM Scaling Laws 00:27:43 Tool Use (Browser, Calculator, Interpreter, DALL-E) 00:33:32 Multimodality (Vision, Audio) 00:35:00 Thinking, System 1/2 00:38:02 Self-improvement, LLM AlphaGo 00:40:45 LLM Customization, GPTs store 00:42:15 LLM OS Part 3: LLM Security 00:45:43 LLM Security Intro 00:46:14 Jailbreaks 00:51:30 Prompt Injection 00:56:23 Data poisoning 00:58:37 LLM Security conclusions End 00:59:23 Outro

You have full access to this article via your institution. An artificial intelligence (AI) system based on Google DeepMind’s AlphaZero AI created algorithms that, when translated into the standard programming language C++, can sort data up to three times as fast as human-generated versions.

On May 30, a research organization called the Center for AI Safety released a 22-word statement signed by a number of prominent "AI scientists," including Sam Altman, the CEO of OpenAI; Demis Hassabis, the CEO of Google DeepMind; and Geoffrey Hinton, who has been described as the "godfather" of AI.

Hi, woke moralists! In today’s episode, we take a look at Jordan Peterson through the lens of Jungian archetypes – just as he would want – and explore some of his other ideas about climate change, gender, marriage, hierarchies, and lobsters. Don’t check the timecode!

Get your BILLIONAIRES ARE NOT YOUR FRIENDS merch here: https://www.teepublic.com/t-shirt/31298912-billionaires-are-not-your-friends?store_id=237592

Check out our new compilation series, CODY COMPS here: https://www.youtube.com/playlist?list=PLqFkH8uXvlJbGeJKUChW4VGxCMhKQ9cJ8

Please fill out our SURVEY: https://kastmedia.com/survey/

Check out our new series SOME THIS! - https://www.youtube.com/playlist?list=PLkJemc4T5NYbcqTbNmyH3uqutwcj8fHf3

Support us on our PATREON: http://patreon.com/somemorenews Check out our MERCH STORE: https://www.teepublic.com/stores/somemorenews?ref_id=9949 SUBSCRIBE to SOME MORE NEWS: https://tinyurl.com/ybfx89rh

Executive Producer - Katy Stoll (@KatyStoll). Written by David Christopher Bell (@MovieHooligan) and Cody Johnston (@drmistercody). Directed by Will Gordh (@will_gordh). Edited by Gregg Meller (@greggmeller). Graphics by F. Clint DeNisco. Head Writer - David Christopher Bell. Producer - Jonathan Harris. Researcher - Marco Siler-Gonzales (@mijo_marco). Associate Producer - Quincy Tucker (@quincyrashard).

Subscribe to the Even More News and SMN audio podcasts here: Apple Podcasts: https://podcasts.apple.com/us/podcast/some-more-news/id1364825229 Spotify: https://open.spotify.com/show/6ebqegozpFt9hY2WJ7TDiA?si=5keGjCe5SxejFN1XkQlZ3w&dl_branch=1 Stitcher: https://www.stitcher.com/show/even-more-news

Find your new favorite shoes for sunny days and upcoming travel at allbirds.com. That’s http://ALLBIRDS.com.

High-performance beauty and skin-care products made with clean, skin-loving ingredients. Right now, you can get 15% off your first order when you visit thrivecausemetics.com/MORENEWS.

Chapters: 00:00 - Intro 8:53 - Jordan Peterson Is Not A Climate Expert 19:24 - Jordan Peterson Demands That You Respect His Bigotry 26:28 - Who, What, Why, and How is Jordan Peterson? 34:56 - The Persona 1:28:44 - The Shadow 2:00:47 - The Anima And Animus 2:34:13 - The Self

Source list: https://docs.google.com/document/d/1ciPKIdTqGdHsxsj-7vjlcnyq4JDYlLBM-UyH3LBi3No/edit

jordanpeterson #jordanpetersonandjoerogan #jordanpetersonquotes #somemorenews

The question of whether a computer program, or a robot, might become sentient has been debated for decades. In science fiction, we see it all the time. The artificial intelligence establishment overwhelmingly considers this prospect something that might happen in the far future, if at all.

The question of whether a computer program, or a robot, might become sentient has been debated for decades. In science fiction, we see it all the time. The artificial intelligence establishment overwhelmingly considers this prospect something that might happen in the far future, if at all.

Human-level artificial intelligence is close to finally being achieved, according to a lead researcher at Google’s DeepMind AI division.

Human-level artificial intelligence is close to finally being achieved, according to a lead researcher at Google’s DeepMind AI division.

Art takes a unique combination of skill, creativity the very human element of aesthetic taste. But what if the visual arts also fall to AI. In this episode we discuss Dall-E 2, a powerful text to image generator that's set to shake things up.

A.I. Playlist: https://youtube.com/playlist?list=PL0iVR8sl9TiXmZUgZSZOpGFouuFNieqgP

My new Spotify album: https://open.spotify.com/album/2mQkQEgmLxCJC8JpcsiF2T?si=560AO6QiQ3ShO3qKBWnRhw

Sign up link here: https://labs.openai.com/waitlist

--- About ColdFusion --- ColdFusion is an Australian based online media company independently run by Dagogo Altraide since 2009. Topics cover anything in science, technology, history and business in a calm and relaxed environment.

» ColdFusion Discord: https://discord.gg/coldfusion » Podcast I Co-host: https://www.youtube.com/channel/UC6jKUaNXSnuW52CxexLcOJg » Twitter | @ColdFusion_TV » Instagram | coldfusiontv » Facebook | https://www.facebook.com/ColdFusioncollective » Podcast Version of Videos: https://open.spotify.com/show/3dj6YGjgK3eA4Ti6G2Il8H https://podcasts.apple.com/us/podcast/coldfusion/id1467404358

ColdFusion Music Channel: https://www.youtube.com/channel/UCGkpFfEMF0eMJlh9xXj2lMw

ColdFusion Merch: INTERNATIONAL: https://store.coldfusioncollective.com/ AUSTRALIA: https://shop.coldfusioncollective.com/

If you enjoy my content, please consider subscribing! I'm also on Patreon: https://www.patreon.com/ColdFusion_TV Bitcoin address: 13SjyCXPB9o3iN4LitYQ2wYKeqYTShPub8

--- "New Thinking" written by Dagogo Altraide --- This book was rated the 9th best technology history book by book authority. In the book you’ll learn the stories of those who invented the things we use everyday and how it all fits together to form our modern world. Get the book on Amazon: http://bit.ly/NewThinkingbook Get the book on Google Play: http://bit.ly/NewThinkingGooglePlay https://newthinkingbook.squarespace.com/about/

Sources:

https://cdn.openai.com/papers/dall-e-2.pdf

https://fortune.com/2022/04/06/openai-dall-e-2-photorealistic-images-from-text-descriptions/

https://www.engadget.com/open-a-is-dall-e-2-produces-fantastical-images-of-most-anything-you-can-imagine-170056814.html

https://www.engadget.com/the-morning-after-open-ai-dall-e-2-images-111840375.html

https://towardsdatascience.com/generating-images-from-prompts-using-clip-and-stylegan-1f9ed495ddda

My Music Channel: https://www.youtube.com/channel/UCGkpFfEMF0eMJlh9xXj2lMw

//Soundtrack//

Burn Water - Nostalgia Dreams

no spirit - leaves covered by snow

Burn Water - Does it Get Easier (clean)

Nils Frahm - You (Teen Daze Rework)

Jon Hopkins - The Wider Sun

Ben Böhmer - Flug & Fall

Kidnap Kid - Moments (feat. Leo Stannard)

Hammock - Wasted We Stared at the Ceiling

Burn Water - Fate

» Music I produce | http://burnwater.bandcamp.com or » http://www.soundcloud.com/burnwater » https://www.patreon.com/ColdFusion_TV » Collection of music used in videos: https://www.youtube.com/watch?v=YOrJJKW31OA

Producer: Dagogo Altraide

My book: https://repeaterbooks.com/product/how-to-philosophize-with-a-hammer-and-sickle/ Audiobook: https://repeaterbooks.com/audiobooks/how-to-philosophize-with-a-hammer-and-sickle-nietzsche-and-marx-for-the-twenty-first-century/ You can also find copies on other websites, including Amazon, Bookshop.org, Blackwell’s, Powell’s, and others. You can also get the eBook, the kindle edition on Amazon, or read it digitally on Google Books. The audiobook is also available on Audible.

My Patreon: https://www.patreon.com/cuck
My Twitter: https://twitter.com/PhilosophyCuck

The two Suicide songs played here are "Frankie Teardrop" and "Rocket USA"

How can authors use AI writing tools like GPT-3? What's the best way to prompt the models to output usable text? Are there copyright issues with this approach? Author Paul Bellow explains how he is using the tools and how authors need to embrace the possibilities rather than reject them.

https://www-thecreativepenn-com.cdn.ampproject.org/v/s/www.thecreativepenn.com/2021/02/26/ai-augmented-author-gpt3/amp/?amp_gsa=1&_js_v=a6&usqp=mq331AQIKAGwASCAAgM=#amp_tf=From %1$s&aoh=16247239328784&csi=0&referrer=https://www.google.com&ampshare=https://www.thecreativepenn.com/2021/02/26/ai-augmented-author-gpt3/

Thanks to Keeps for sponsoring this video! Head to https://keeps.com/nickrobinson to learn more and get 50% off your first order of hair loss treatment. Check out the bonus features playlist - with deleted scenes, director's commentary and more: https://www.youtube.com/playlist?list=PLGFiGO64XRnjhzze6RlQxWnpcbCJwrubX

Additional editing by f4mi: https://www.youtube.com/channel/UCSWMraguMlNanVQgseTTr_Q

here's my instagram: http://instagram.com/babylonian here's my twitter: http://twitter.com/babylonian follow me on twitch! http://twitch.tv/babylonian subscribe: https://www.youtube.com/user/babylonian?sub_confirmation=1

Google Maps link: https://goo.gl/maps/mnGGNKGsndUVoN2AA

Music: MR carrotV2 - Windows 95 Dnb remix (attempt 1) Yoshinori Adachi - イースII「MOAT OF BURNEDBLESS 1998」（アレンジ曲 Torii Wolf - Lantern Room (Instrumental Version) Marc Russo - Groceries (The Sims original soundtrack) Peelander-Z - So Many Mike Matt Large - Believe Me When I Say It Yomoti - Before Chill Kikoru - Happytown Junichi Aoki - Temporary Peace (PC88 version) STRLGHT - Hype Age of Empires II - Shamburger Flat Theory - Outer Circle William Benckert - Clouded Mind Google Earth VR OST Peter Sandberg - How Peculiar Peter Sandberg - Butterflies Blue Steel - R&R Cora Zea - Elderly Melodia Ruben Cortes - Canta Canta Peter Sandberg - Butterflies

Want to directly support more videos like this one? Consider joining my channel! https://www.youtube.com/channel/UCOuddH5GyBXp-_tv_ASdp_A/join

Search Google Images for ‘counterculture’ and it overwhelmingly returns black-and-white photos of young people all now over 60.

“If you look at Google Earth, it’s springtime everywhere,” explains Gopal Shah, Google Earth’s product manager, in a YouTube interview. In a TED talk he boasts that Google Earth is “cloud-free,” since the clouds and their shadows are edited out.

List of AI in: Academia, Technology and Activism - An Openly shared Resource with Peers. (Title to be changed soon) The Cultural Production of Art and AI: Investigating various methods such as: computer vision, artificial intelligence, neurorobotics, speech recognition, generative writing, ge...

At its core, artificial intelligence is a military technology. Why is the company sharing it with a rival? Mr. Thiel is an entrepreneur and investor.