Humans have always lived alongside monsters, both real and imagined. The etymological root of the word also gives us “demonstrate,” and monsters have always been used by humans to demonstrate the same thing: what is not us.

Welcome to the Convivial Society, a newsletter about technology and culture.

How can a person make up for seven decades of misrepresentation and willful distortion in the time allotted to a sound bite? Recently, an Australian-Palestinian friend of mine was invited to appear on Australia’s national television network to discuss the situation in and around Gaza.

Conspiracy theories are popular and there is no doubt that the internet has fuelled them on. From the theory that 9/11 was an inside job to the idea that reptilian humanoids rule the world, conspiracy theories have found a natural home online.

Whatever future comes, one thing is fairly certain, excepting total American collapse—the goods will continue to flow. By human, by vehicle, by automaton, or by some combination of the three,stuff will be distributed, sold, and speculated upon across the land.

Of all the buzzy 21st-century tech phrases, “machine learning” threatens to be the most important. Programming computers is slow, but we’re nearing the point where humans give the bots parameters and let them teach themselves.

Undiagnosed by the mainstream media and much of the academic community, a major intellectual renewal is underway across the left. It is energetic and tech-savvy, building platforms such as Novara Media.

And a chatbot is not a human. And a linguist named Emily M. Bender is very worried what will happen when we forget this. This article was featured in One Great Story, New York’s reading recommendation newsletter. Sign up here to get it nightly. Nobody likes an I-told-you-so.

This is a 1 hour general-audience introduction to Large Language Models: the core technical component behind systems like ChatGPT, Claude, and Bard. What they are, where they are headed, comparisons and analogies to present-day operating systems, and some of the security-related challenges of this new computing paradigm. As of November 2023 (this field moves fast!).

Context: This video is based on the slides of a talk I gave recently at the AI Security Summit. The talk was not recorded but a lot of people came to me after and told me they liked it. Seeing as I had already put in one long weekend of work to make the slides, I decided to just tune them a bit, record this round 2 of the talk and upload it here on YouTube. Pardon the random background, that's my hotel room during the thanksgiving break.

• Slides as PDF: https://drive.google.com/file/d/1pxx_ZI7O-Nwl7ZLNk5hI3WzAsTLwvNU7/view?usp=share_link (42MB)
• Slides. as Keynote: https://drive.google.com/file/d/1FPUpFMiCkMRKPFjhi9MAhby68MHVqe8u/view?usp=share_link (140MB)

Few things I wish I said (I'll add items here as they come up): - The dreams and hallucinations do not get fixed with finetuning. Finetuning just "directs" the dreams into "helpful assistant dreams". Always be careful with what LLMs tell you, especially if they are telling you something from memory alone. That said, similar to a human, if the LLM used browsing or retrieval and the answer made its way into the "working memory" of its context window, you can trust the LLM a bit more to process that information into the final answer. But TLDR right now, do not trust what LLMs say or do. For example, in the tools section, I'd always recommend double-checking the math/code the LLM did. - How does the LLM use a tool like the browser? It emits special words, e.g. |BROWSER|. When the code "above" that is inferencing the LLM detects these words it captures the output that follows, sends it off to a tool, comes back with the result and continues the generation. How does the LLM know to emit these special words? Finetuning datasets teach it how and when to browse, by example. And/or the instructions for tool use can also be automatically placed in the context window (in the “system message”). - You might also enjoy my 2015 blog post "Unreasonable Effectiveness of Recurrent Neural Networks". The way we obtain base models today is pretty much identical on a high level, except the RNN is swapped for a Transformer. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ - What is in the run.c file? A bit more full-featured 1000-line version hre: https://github.com/karpathy/llama2.c/blob/master/run.c

Chapters: Part 1: LLMs 00:00:00 Intro: Large Language Model (LLM) talk 00:00:20 LLM Inference 00:04:17 LLM Training 00:08:58 LLM dreams 00:11:22 How do they work? 00:14:14 Finetuning into an Assistant 00:17:52 Summary so far 00:21:05 Appendix: Comparisons, Labeling docs, RLHF, Synthetic data, Leaderboard Part 2: Future of LLMs 00:25:43 LLM Scaling Laws 00:27:43 Tool Use (Browser, Calculator, Interpreter, DALL-E) 00:33:32 Multimodality (Vision, Audio) 00:35:00 Thinking, System 1/2 00:38:02 Self-improvement, LLM AlphaGo 00:40:45 LLM Customization, GPTs store 00:42:15 LLM OS Part 3: LLM Security 00:45:43 LLM Security Intro 00:46:14 Jailbreaks 00:51:30 Prompt Injection 00:56:23 Data poisoning 00:58:37 LLM Security conclusions End 00:59:23 Outro

Scientists are planning to solve one of the world’s greatest archaeological mysteries.

As the alarm bells have made it urgently clear—humanity has breached planetary boundaries—causing anthropogenic climate change and environmental disaster across the world.

M John Harrison, 77, is the author of seven story collections and 12 novels, including Nova Swing, which won the Arthur C Clarke award for science fiction in 2007, and The Sunken Land Begins to Rise Again, winner of the Goldsmiths prize in 2020.

You have full access to this article via your institution. An artificial intelligence (AI) system based on Google DeepMind’s AlphaZero AI created algorithms that, when translated into the standard programming language C++, can sort data up to three times as fast as human-generated versions.

On May 30, a research organization called the Center for AI Safety released a 22-word statement signed by a number of prominent "AI scientists," including Sam Altman, the CEO of OpenAI; Demis Hassabis, the CEO of Google DeepMind; and Geoffrey Hinton, who has been described as the "godfather" of AI.

A dazzlingly fast chatbot cannot replace the authentic and reflective voice of a thinking, living human being.

No one can define it, much less fully replicate it. If an alien life form landed on Earth tomorrow and called up some of the planet’s foremost experts on lactation, it would have a heck of time figuring out what, exactly, humans and other mammals are feeding their kids.

Among the more sombre gifts brought by the Enlightenment was the realisation that humans might one day become extinct.

Can humanity turn back from the brink? Croatian philosopher Srećko Horvat on breaking the spell of end-times nihilism.

https://novaramedia.com/2022/04/06/how-we-normalised-the-apocalypse-with-srecko-horvat/

Big ideas reflecting our evolution and what makes us intrinsically human.

Decadent art is the flower of late civilizations, the sign of the end lies within its apocalyptic beauty which offers only the most morbid and beautiful enrichments, an excess of sensual cruelty that annihilates self and other in a languid tempest of enervating desires.