Large language models like those offered by OpenAI and Google famously require vast troves of training data to work. The latest versions of these models have already scoured much of the existing internet which has led some to fear there may not be enough new data left to train future iterations.

It is estimated that one language dies every 14 days, with it a world of concepts and ideas that could only be expressed in its words. Most of the languages at risk of extinction belong to Indigenous communities.

This video explores the journey of AI language models, from their modest beginnings through the development of OpenAI's GPT models. Our journey takes us through the key moments in generative neural network research involved in next word prediction. We delve into the early experiments with tiny language models in the 1980s, highlighting significant contributions by researchers like Jordan, who introduced Recurrent Neural Networks, and Elman, whose work on learning word boundaries revolutionized our understanding of language processing. It leaves us with a question: what is thought? Is simulated thought, thought? Featuring Noam Chomsky Douglas Hofstadter Michael I. Jordan Jeffrey Elman Geoffrey Hinton Ilya Sutskever Andrej Karpathy Yann LeCun and more. (Sam altman)

My script, references & visualizations here: https://docs.google.com/document/d/1s7FNPoKPW9y3EhvzNgexJaEG2pP4Fx_rmI4askoKZPA

consider joining my channel as a YouTube member: https://www.youtube.com/channel/UCotwjyJnb-4KW7bmsOoLfkg/join

This is the last video in the series "The Pattern Machine" you can watch it all here: https://www.youtube.com/playlist?list=PLbg3ZX2pWlgKV8K6bFJr5dhM7oOClExUJ

00:00 - Introduction 00:32 - hofstader's thoughts on chatGPT 01:00 - recap of supervised learning 01:55 - first paper on sequential learning 02:55 - first use of state units (RNN) 04:33 - first observation of word boundary detection 05:30 - first observation of word clustering 07:16 - first "large" language model Hinton/Sutskever 10:10 - sentiment neuron (Ilya | OpenAI) 12:30 - transformer explaination 15:50 - GPT-1 17:00 - GPT-2 17:55 - GPT-3 18:20 - In-context learning 19:40 - ChatGPT 21:10 - tool use 23:25 - philosophical question: what is thought?

But even after I overcame my instinct for detachment, I remained wary of the movement’s language, which was a language of binaries: women and men.

This is a 1 hour general-audience introduction to Large Language Models: the core technical component behind systems like ChatGPT, Claude, and Bard. What they are, where they are headed, comparisons and analogies to present-day operating systems, and some of the security-related challenges of this new computing paradigm. As of November 2023 (this field moves fast!).

Context: This video is based on the slides of a talk I gave recently at the AI Security Summit. The talk was not recorded but a lot of people came to me after and told me they liked it. Seeing as I had already put in one long weekend of work to make the slides, I decided to just tune them a bit, record this round 2 of the talk and upload it here on YouTube. Pardon the random background, that's my hotel room during the thanksgiving break.

• Slides as PDF: https://drive.google.com/file/d/1pxx_ZI7O-Nwl7ZLNk5hI3WzAsTLwvNU7/view?usp=share_link (42MB)
• Slides. as Keynote: https://drive.google.com/file/d/1FPUpFMiCkMRKPFjhi9MAhby68MHVqe8u/view?usp=share_link (140MB)

Few things I wish I said (I'll add items here as they come up): - The dreams and hallucinations do not get fixed with finetuning. Finetuning just "directs" the dreams into "helpful assistant dreams". Always be careful with what LLMs tell you, especially if they are telling you something from memory alone. That said, similar to a human, if the LLM used browsing or retrieval and the answer made its way into the "working memory" of its context window, you can trust the LLM a bit more to process that information into the final answer. But TLDR right now, do not trust what LLMs say or do. For example, in the tools section, I'd always recommend double-checking the math/code the LLM did. - How does the LLM use a tool like the browser? It emits special words, e.g. |BROWSER|. When the code "above" that is inferencing the LLM detects these words it captures the output that follows, sends it off to a tool, comes back with the result and continues the generation. How does the LLM know to emit these special words? Finetuning datasets teach it how and when to browse, by example. And/or the instructions for tool use can also be automatically placed in the context window (in the “system message”). - You might also enjoy my 2015 blog post "Unreasonable Effectiveness of Recurrent Neural Networks". The way we obtain base models today is pretty much identical on a high level, except the RNN is swapped for a Transformer. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ - What is in the run.c file? A bit more full-featured 1000-line version hre: https://github.com/karpathy/llama2.c/blob/master/run.c

Chapters: Part 1: LLMs 00:00:00 Intro: Large Language Model (LLM) talk 00:00:20 LLM Inference 00:04:17 LLM Training 00:08:58 LLM dreams 00:11:22 How do they work? 00:14:14 Finetuning into an Assistant 00:17:52 Summary so far 00:21:05 Appendix: Comparisons, Labeling docs, RLHF, Synthetic data, Leaderboard Part 2: Future of LLMs 00:25:43 LLM Scaling Laws 00:27:43 Tool Use (Browser, Calculator, Interpreter, DALL-E) 00:33:32 Multimodality (Vision, Audio) 00:35:00 Thinking, System 1/2 00:38:02 Self-improvement, LLM AlphaGo 00:40:45 LLM Customization, GPTs store 00:42:15 LLM OS Part 3: LLM Security 00:45:43 LLM Security Intro 00:46:14 Jailbreaks 00:51:30 Prompt Injection 00:56:23 Data poisoning 00:58:37 LLM Security conclusions End 00:59:23 Outro

This is a guest post by Daisy Dixon, Lecturer in Philosophy at Cardiff University Content warning: This post includes discussion of abusive language, misogyny, body-shaming, and rape threats.

You have full access to this article via your institution. An artificial intelligence (AI) system based on Google DeepMind’s AlphaZero AI created algorithms that, when translated into the standard programming language C++, can sort data up to three times as fast as human-generated versions.

Workers hired via crowdsource services like Amazon Mechanical Turk are using large language models to complete their tasks – which could have negative knock-on effects on AI models in the future. Data is critical to AI.

Recorded Monday June 20th, 2022

After fifteen years art directing photography books and magazines such as Elle and Tatler, Dr Grant Scott began to work solely as a photographer for commercial and editorial clients in 2000. His images combine his experience working with some of the greatest photographers of the last century with his graphic and journalistic talents to create engaging photographic narratives from every commission.

Grant is the founder/curator of United Nations of Photography, a Senior Lecturer and Subject Coordinator for Photography at Oxford Brookes University, Oxford, a working photographer, documentary filmmaker, BBC Radio contributor and the author of Professional Photography: The New Global Landscape Explained (Routledge 2014), The Essential Student Guide to Professional Photography (Routledge 2015) and New Ways of Seeing: The Democratic Language of Photography (Routledge 2019).

In the talk Grant addresses the constantly shifting sands of engagement with the photographic medium. Noting that we read books and acquire a respect for how difficult it is to write and are taught how to read the written word and appreciate its transformative power Grant will makes the case for the importance of visual literacy. He will explain how digital photography has democratised the medium, with ensuing debates between those with analogue experience and the digital natives and focus on the importance of ‘the personal’ in defining subject matter. The talk and following questions and answers are an exploration on what makes a successful visual narrative.

To hear more audio stories from publications like The New York Times, download Audm for iPhone or Android. You are sitting in a comfortable chair by the fire, on a cold winter’s night. Perhaps you have a mug of tea in hand, perhaps something stronger.

To hear more audio stories from publications like The New York Times, download Audm for iPhone or Android. You are sitting in a comfortable chair by the fire, on a cold winter’s night. Perhaps you have a mug of tea in hand, perhaps something stronger.

A text written for IOCOSE‘s exhibition “All of Your Base” held at Aksioma Project Space in Ljubljana, 1 December 2021–14 January 2022.

Published as a PostScriptUM PDF and print-on-demand publication.

► WHY I WANT TO FUCK ELON MUSK ► eBROCHURE (PDF)► PRINT ON DEMAND [coming soon]► LIST ON ISSUU

In their space race the gurus of the NewSpace movement are expanding an imaginary that hybridizes individualism, libertarianism, neoliberal economics, counterculture and utopianism. “Why I Want to Fuck Elon Musk” plays with these cultural references, taking inspiration from the most emblematic statements spoken or tweeted by Elon Musk in recent years. Daniel Rourke, a London-based writer, artist and academic, has resorted to working with the OpenAI Generative Pre-trained Transformer 3 (GPT-3) language model to imagine and narrate chronicles from a near future in which blockchains have materialized and the deepfakes of Bezos and Musk have colonized Mars. The fictional universe thus created by human and non-human imagination builds a literary counterpart to IOCOSE’s latest works – the video animations Pointing at a New Planet (2020) and Free from History (2021) – presented on the occasion of the “All of Your Base” exhibition at Aksioma | Project Space in Ljubljana.

Download PDF (eBROCHURE) View on Aksioma website

We worked with Keaton Patti to make a bot watch over 400,000 hours of horror movies and then write its own horror movie. This is what it came up with.

NetflixByBots #KeatonPatti #Horror

Subscribe: https://bit.ly/2Kncxw6

About Netflix Is A Joke: The official hub of Netflix stand-up, comedy series, films, and all things funny — curated by the world’s most advanced algorithm and a depressed, yet lovable, cartoon horse. Their unlikely friendship is our story…

About Netflix: Netflix is the world's leading streaming entertainment service with over 209 million paid memberships in over 190 countries enjoying TV series, documentaries and feature films across a wide variety of genres and languages. Members can watch as much as they want, anytime, anywhere, on any internet-connected screen. Members can play, pause and resume watching, all without commercials or commitments.

Connect with Netflix Is A Joke: Visit Netflix WEBSITE: http://nflx.it/29BcWb5 Like Netflix Is A Joke on FACEBOOK: https://bit.ly/2xD5zfu Follow Netflix Is A Joke on TWITTER: https://bit.ly/2N6ENkx Follow Netflix Is A Joke on INSTAGRAM: https://bit.ly/2VTQcxu

In this solo episode, I discuss the impact of converging technologies, Artificial Intelligence, Natural Language Generation (NLG) tools like GPT-3, and more on writing, authors, and the publishing industry.

Te Hiku Media gathered huge swathes of Māori language data. Corporates are now trying to get the rights to it

submitted by /u/DukkyDrake to r/singularity [link] [comments]

AI Dictionary is a Twitter bot that tweets a . I wanted to see how much OpenAI's language model actually knew, so I tried to get it to define words... but I accidentally had it set to 'random', not 'best'. [via mefi projects] Semi-related: OpenAI's GPT-2: the model, the hype, and the controversy (Towards Data Science), and Experimenting with OpenAI's Improved Language Model (short post on Medium), which notes "The public at large will need to become more skeptical of the content they consume online."

Goldsmiths College Department of Art MFA Lectures 2018 - 2019

Series 1.1: Offence is the Best Defence: On the Success of Social Media Toxicity

8 Oct 2018 — Daniel Rourke (Goldsmiths): “We're trying to have the non-weird future get here as fast as possible.” 15 Oct 2018 — Isobelle Clarke (Birmingham): "Poor little snowflake, are you 'grossly' offended?": Quantifying Communicative Styles of Twitter Trolling 22 Oct 2018 — Zeena Feldman (Kings College, London): Beyond Time: On Quitting Social Media 29 Oct 2018 — William Davies (Goldsmiths): War of Words: Embodiment and Rhetoric in Online Combat

Daniel Rourke 8th October 2018 “We're trying to have the non-weird future get here as fast as possible.”

From the Latin ‘aequivocare’, for ‘called by the same name’, to equivocate is to use language ambiguously to conceal a truth or avoid commitment to a single meaning. In this talk Daniel Rourke will consider equivocation in the performative (social media) speech acts of figures such as Donald Trump, Elon Musk, and Twitter CEO Jack Dorsey.

How their speech acts exposit a 'shared' future, or a means of ‘escaping’ our present conditions, has much to tell us about how the very idea of the ‘true’ or the ‘false’ has shifted in the era of algorithmic governance, and social media campaigns such as #MeToo.

Turning to Homi K. Bhabha's theories of postcolonial discourse, as well as introducing the project The 3D Additivist Manifesto – co-created with Morehshin Allahyari – Daniel will end by trying to reaffirm the equivocal act, pointing out a way to generate and move toward non-determinate futures without imperialising them.

BIO: Dr. Daniel Rourke is a writer/artist and co-convener of Digital Media (MA) at Goldsmiths. In his work Daniel creates collaborative frameworks and theoretical toolsets for exploring the intersection of digital materiality, the arts, and posthumanism. These frameworks often hinge on speculative elements taken from science fiction and pop culture: fictional figures and fabulations that might offer a glimpse of a radical ‘outside’ to the human(ities). His writing and artistic profile includes work with AND Festival, The V&A, FACT Liverpool, Arebyte gallery, Centre Pompidou, Transmediale, Tate Modern, Sonic Acts Festival, as well as recent artistic collaborations with a cast of hundreds... web: machinemachine.net.

Presented by the Art Department, Goldsmiths.

Humans are the only Earthlings with complex language. But at what cost was that ability acquired? In this episode, I visit Tetsuro Matsuzawa to learn about his influential cognitive tradeoff hypothesis.

Available with YouTube Premium - https://www.youtube.com/premium/originals. To see if Premium is available in your country, click here: https://goo.gl/A3HtfP

RISING Dispatches from the New American Shore By Elizabeth Rush Illustrated. 299 pp. Milkweed Editions. $26. A drowned world: It’s an ancient fear and a very old story.