Neural Networks – Intuitively and Exhaustively Explained

Welcome to the first of what we hope will be many posts about teaching writing in the age of AI! The Important Work is a space for writing instructors at all levels—high school, college, and beyond—to share reflections about teaching writing in the era of generative AI.

How can a person make up for seven decades of misrepresentation and willful distortion in the time allotted to a sound bite? Recently, an Australian-Palestinian friend of mine was invited to appear on Australia’s national television network to discuss the situation in and around Gaza.

Neuroscience research into people with aphantasia, who don’t experience mental imagery, is revealing how imagination works and demonstrating the sweeping variety in our subjective experiences. Introduction Two years ago, Sarah Shomstein realized she didn’t have a mind’s eye.

Large language models like those offered by OpenAI and Google famously require vast troves of training data to work. The latest versions of these models have already scoured much of the existing internet which has led some to fear there may not be enough new data left to train future iterations.

Daniël de Zeeuw: A combination of genuine concerns and moral panics over AI (often fueled by the tech moguls themselves) has reached a new high in recent years, particularly with the introduction of ChatGPT by OpenAI.

Hex makes a living in virtual reality. She’s an online sex worker, hosting shows and posting photos and videos from social VR platform VRChat to her Fansly account, a subscription site for erotic content.

On December 15, 1811, the London Statesman issued a warning about the state of the stocking industry in Nottingham. Twenty thousand textile workers had lost their jobs because of the incursion of automated machinery.

This video explores the journey of AI language models, from their modest beginnings through the development of OpenAI's GPT models. Our journey takes us through the key moments in generative neural network research involved in next word prediction. We delve into the early experiments with tiny language models in the 1980s, highlighting significant contributions by researchers like Jordan, who introduced Recurrent Neural Networks, and Elman, whose work on learning word boundaries revolutionized our understanding of language processing. It leaves us with a question: what is thought? Is simulated thought, thought? Featuring Noam Chomsky Douglas Hofstadter Michael I. Jordan Jeffrey Elman Geoffrey Hinton Ilya Sutskever Andrej Karpathy Yann LeCun and more. (Sam altman)

My script, references & visualizations here: https://docs.google.com/document/d/1s7FNPoKPW9y3EhvzNgexJaEG2pP4Fx_rmI4askoKZPA

consider joining my channel as a YouTube member: https://www.youtube.com/channel/UCotwjyJnb-4KW7bmsOoLfkg/join

This is the last video in the series "The Pattern Machine" you can watch it all here: https://www.youtube.com/playlist?list=PLbg3ZX2pWlgKV8K6bFJr5dhM7oOClExUJ

00:00 - Introduction 00:32 - hofstader's thoughts on chatGPT 01:00 - recap of supervised learning 01:55 - first paper on sequential learning 02:55 - first use of state units (RNN) 04:33 - first observation of word boundary detection 05:30 - first observation of word clustering 07:16 - first "large" language model Hinton/Sutskever 10:10 - sentiment neuron (Ilya | OpenAI) 12:30 - transformer explaination 15:50 - GPT-1 17:00 - GPT-2 17:55 - GPT-3 18:20 - In-context learning 19:40 - ChatGPT 21:10 - tool use 23:25 - philosophical question: what is thought?

What Are You Watching? is a weekly space for The A.V Club’s staff and readers to share their thoughts, observations, and opinions on film and TV.  It’s common for viewers first coming to the work of the late Polish director Andrzej Żuławski to find it both wildly undisciplined and inspired.

Warning: Contains strobing effects. A short animated documentary by Eryk Salvaggio on how AI image generation works, and the entanglement of composite photography,…

This is a 1 hour general-audience introduction to Large Language Models: the core technical component behind systems like ChatGPT, Claude, and Bard. What they are, where they are headed, comparisons and analogies to present-day operating systems, and some of the security-related challenges of this new computing paradigm. As of November 2023 (this field moves fast!).

Context: This video is based on the slides of a talk I gave recently at the AI Security Summit. The talk was not recorded but a lot of people came to me after and told me they liked it. Seeing as I had already put in one long weekend of work to make the slides, I decided to just tune them a bit, record this round 2 of the talk and upload it here on YouTube. Pardon the random background, that's my hotel room during the thanksgiving break.

• Slides as PDF: https://drive.google.com/file/d/1pxx_ZI7O-Nwl7ZLNk5hI3WzAsTLwvNU7/view?usp=share_link (42MB)
• Slides. as Keynote: https://drive.google.com/file/d/1FPUpFMiCkMRKPFjhi9MAhby68MHVqe8u/view?usp=share_link (140MB)

Few things I wish I said (I'll add items here as they come up): - The dreams and hallucinations do not get fixed with finetuning. Finetuning just "directs" the dreams into "helpful assistant dreams". Always be careful with what LLMs tell you, especially if they are telling you something from memory alone. That said, similar to a human, if the LLM used browsing or retrieval and the answer made its way into the "working memory" of its context window, you can trust the LLM a bit more to process that information into the final answer. But TLDR right now, do not trust what LLMs say or do. For example, in the tools section, I'd always recommend double-checking the math/code the LLM did. - How does the LLM use a tool like the browser? It emits special words, e.g. |BROWSER|. When the code "above" that is inferencing the LLM detects these words it captures the output that follows, sends it off to a tool, comes back with the result and continues the generation. How does the LLM know to emit these special words? Finetuning datasets teach it how and when to browse, by example. And/or the instructions for tool use can also be automatically placed in the context window (in the “system message”). - You might also enjoy my 2015 blog post "Unreasonable Effectiveness of Recurrent Neural Networks". The way we obtain base models today is pretty much identical on a high level, except the RNN is swapped for a Transformer. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ - What is in the run.c file? A bit more full-featured 1000-line version hre: https://github.com/karpathy/llama2.c/blob/master/run.c

Chapters: Part 1: LLMs 00:00:00 Intro: Large Language Model (LLM) talk 00:00:20 LLM Inference 00:04:17 LLM Training 00:08:58 LLM dreams 00:11:22 How do they work? 00:14:14 Finetuning into an Assistant 00:17:52 Summary so far 00:21:05 Appendix: Comparisons, Labeling docs, RLHF, Synthetic data, Leaderboard Part 2: Future of LLMs 00:25:43 LLM Scaling Laws 00:27:43 Tool Use (Browser, Calculator, Interpreter, DALL-E) 00:33:32 Multimodality (Vision, Audio) 00:35:00 Thinking, System 1/2 00:38:02 Self-improvement, LLM AlphaGo 00:40:45 LLM Customization, GPTs store 00:42:15 LLM OS Part 3: LLM Security 00:45:43 LLM Security Intro 00:46:14 Jailbreaks 00:51:30 Prompt Injection 00:56:23 Data poisoning 00:58:37 LLM Security conclusions End 00:59:23 Outro

Become a Patron https://www.patreon.com/bobbyfingers

Or you can buy me a coffee here. https://ko-fi.com/bobbyfingers

UPDATE : THE TREASURE HAS BEEN FOUND BY A PATRON!!!

Produced by Rob Sutton and Bobby Fingers.

Thanks to the good people of ODYSSEY STUDIOS https://www.odysseystudios.ie/

Click for KING KONG COMPANY https://youtu.be/_A13Mv5dchI?si=C2wBpJBQPnsj4dr8

Big thanks to L. Sierakowski for the use of his beautiful music. https://furniture-music.bandcamp.com

Thanks to Esref Akinci, Ossman Yetkin and Dr. Nihat Cicek at Holiday Estetic https://holidayestetic.com/

Amazon march footage courtesy of Shabbir Lakha/Rank and FIle Combine https://www.instagram.com/shabbirlakha/ CHECK HIM OUT!!!! AMAZING PERSON!!!

Director of Photography - Shane Serrano - https://www.shaneserrano.com/

Additional Camera work by Rob Sutton https://www.instagram.com/rob.sutton/ Clem McInerney (Water Cinematography) https://clemmcinerney.com/ Shane Joyce https://www.instagram.com/shanej0yce/ Alex Genn-Bash https://www.instagram.com/shuttermug/ Berke Beyazpınar https://www.instagram.com/berkebeyazpinar/ Emma McNamara https://www.instagram.com/rc86maker/ Jason Flood (whats your social Jason?)

Music by Bobby Fingers Cillian King https://www.instagram.com/cillianking/ Daniel luke https://open.spotify.com/artist/7MqR9hflQxb3YzurijtPB3 Luís Sierakowski https://furniture-music.bandcamp.com Boris Sevastyanov https://www.fiverr.com/composerr Michael Hennessy and all the guys in Harmony Bro, Mens Choir in Ennis.

Sound Recordist - Eoin Gleeson https://www.instagram.com/eoingleeson/

Visual Effects by Ray Sullivan https://www.instagram.com/directedbyray/

Additional editing by Rob Sutton and Nathan Campion https://www.instagram.com/nathanthecampionoftheworld/

Makeup by Sadhbh Nic Uidhir https://www.instagram.com/sadhbhniqueer/ Clodagh McInerney

Costumes by Gerdi McGlynn Pamela McGlynn Grace Burke Sarah Allen

Jet Ski Operator - James Monaghan

The Pirates were played by Jean McGlynn Michael O'Donoghue Mike Finnernanan Martin Cooke Nhlanhla Banda Cein Daly Shane Davis Luke McEvoy Isaac Bryant Cian Egan Matt Kelly Darren O'Carroll Tobi Omoteso Conor McNelis Robert Walsh

Special thanks to Shirani Bolle, Díní McGlynn, Romee Mcglynn, Antoinette Bolle, Wayne Cuddy, Dominic Yard http://www.thisisyard.com/ Ali Day, Rebecca Ní Churnáin, Mick Dolan and all the gang at Dolans, Rebecca Pearson, Bec Hill, Matt Parker, Dr. Billy Fingers, Gary MacMahon, Brian O'Brien, Mike Hogan, Bono Kavanagh, Emma Tierney, Steve Lock and the gang at Soho theatre, Conor O'Brien, Laura Davoren, Paul Ryan from Film In Limerick, Keelan Christie, Mary Gallagher Cooke, Brendan Kennedy from Panda.ie. Thanks to John and all the lads in Mungret Civic Ameneties, Elle Coughlan, Alan Ward and the guys in the National 50m Pool in UL.

If I've forgotten anyone, please let me know, I can add your name in here very easily.

The bartender pouring free drinks glances around with nothing to do as the raucous room full of startup founders and software developers gravitates toward its preferred drug: networking.

A video about neural networks, function approximation, machine learning, and mathematical building blocks. Dennis Nedry did nothing wrong. This is a submission for #SoME3

My Links Patreon: https://www.patreon.com/emergentgarden Discord: https://discord.gg/ZsrAAByEnr

Links and Content: On Mathematical Maturity, Thomas Garrity: https://www.youtube.com/watch?v=zHU1xH6Ogs4 Earth Rotation Loop: https://www.youtube.com/watch?v=aiQdLP2mBJE Modeling Shell Surfaces: https://www.geogebra.org/m/xtv7zpn5 Fourier Features Paper: https://arxiv.org/abs/2006.10739 Code for mandelbrot/image approximations: https://github.com/MaxRobinsonTheGreat/mandelbrotnn Code for line/surface approximations: https://github.com/MaxRobinsonTheGreat/ManimApproximations

Music: https://youtube.com/@acolyte-compositions

Timestamps (0:00) Functions Describe the World (3:15) Neural Architecture (5:35) Higher Dimensions (11:55) Taylor Series (15:20) Fourier Series (21:25) The Real World (24:32) An Open Challenge

Responding to the 1 millionth time that people have said "the left has failed men" and somehow... kinda agreeing with Shoe on Head

2:00 Y'all are overreacting to that table 7:40 The Manosphere isn't new it's just more visible 11:08 Algorithms make this an uphill battle 15:14 The online left does have a weak analysis of masculinity 17:57 Stop whining about mean feminist and do the work 19:40 The real problems facing men

Classic debunks of various manosphere topics MuneCat debunks the manosphere - https://www.youtube.com/watch?v=BgO25FTwfRI&pp=ygUHbXVuZWNhdA%3D%3D My Black Manosphere video - https://www.youtube.com/watch?v=upt_ks61_70&t=1466s&pp=ygUQYmxhY2sgbWFub3NwaGVyZQ%3D%3D Sissyphus trying to make sense of the manosphere - https://www.youtube.com/watch?v=lSq3bcyrJY0&pp=ygUOdGhlIG1hbm9zcGhlcmU%3D Frank Laundry explaining Fresh and Fit - https://www.youtube.com/watch?v=WBsCmYCShqI&pp=ygUbZnJhbmsgbGF1bmRyeSBmcmVzaCBhbmQgZml0 Zatzman on the Manosphere as a content farm - https://www.youtube.com/watch?v=pvZITKjaffw&pp=ygUOdGhlIG1hbm9zcGhlcmU%3D

Other male content creators to check out

@Salari @FinntasticMrFox @ForeignManinaForeignLand @NoahSamsen @Sisyphus55 @TheZatzman @COLORMIND.mp4 @victorythecreator @lilbilliam @ThinkpieceTribe @JawnLouis

Monitoring the future study 2022 - https://monitoringthefuture.org/wp-content/uploads/2022/12/mtf2022.pdf Political ideology data from 2021 - https://monitoringthefuture.org/wp-content/uploads/2023/08/PoliticalBeliefTrends12thGraders1976-2021.xlsx CDC Deaths of despair study - https://www.cdc.gov/surveillance/blogs-stories/deaths-of-dispair.html Meta study mentioned - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9566538/

Peter Bauman (Monk Antony) develops a framework for appreciating and analyzing generative art through an investigation of the movement’s locus of artistic intent.

This field guide looks at social media that works on different “logics” than do Facebook, Twitter, or YouTube. It features communities that have tried consciously to use different models than surveillance capitalism and includes the work of collaborators in other countries and subcultures.

Workers hired via crowdsource services like Amazon Mechanical Turk are using large language models to complete their tasks – which could have negative knock-on effects on AI models in the future. Data is critical to AI.

Guardian: 'The annual CGC (Crap Games Competition) has been bringing Spectrum fans together for more than 25 years' ... "What makes the CGC entertaining is the self-deprecating, sardonic British humour," explains 43-year-old Paul Collins from Reading, who first entered the CGC in 2000 with Pear-Shaped ("a simple maze game where you try to collect as many pears as possible") and Crap Football, featuring a digitised Des Lynam. "There are ideas that can't possibly work, eg Sim City: The Text Adventure or Blind Flight Simulator. Or names that are just funny, like Whack a Nun II and European Sandwich Hunt."