A fundamental flaw leaves LLMs strikingly vulnerable to attack - MIT Technology Review

On an afternoon in Seoul in March 2016, I watched a program I helped build put a stone on the fifth line of a Go board in what looked like a gift to its human opponent. Move 37 in game two of the five-game match looked so absurd that some commentators thought it was a programming glitch. It wasn’t. AlphaGo won the game, ultimately triumphing 4-1 over Lee Sedol, one of the greatest professional Go players of all time. “I thought AlphaGo was based on probability calculation and that it was merely a machine,” Lee said afterwards. “But when I saw this move, I changed my mind. Surely, AlphaGo is creative.”  When Deep Blue defeated then reigning world chess champion Garry Kasparov in 1997, it did so by looking six to eight moves ahead per player and evaluating 200 million chess positions per second, using rules hard-coded by humans. Go is a vastly more complex game. A stone’s worth depends on how distant groups and territory unfold over dozens of moves. Computing even a fraction of the possible outcomes would take a supercomputer billions of years. To win, AlphaGo had to sense who was ahead at a glance and even invent moves no human had thought to play.

Software only has value when shipped; faster release cadence matters. Development, deployment, and operations require different thinking, metrics, and investments, Titus Winters explained in his talk Software Workflow Optimization at Craft Conference. He mentioned four impacts in tech: product success, compute capacity, engineering capacity, and strategic innovation. The biggest expense is people, not compute, which goes mostly unmeasured. AI shifts engineers toward deeper problem understanding, not fewer engineers, he argued. Developing software with or without AI, we have to carefully reason about the goals, inputs, and outputs of each phase of the software workflow, Winters mentioned. He referred to the paper Develop, Deploy, Operate - a holistic model for understanding the costs and value of software development that he wrote with Leah Rivers and Salim Virji, which provides guidance for reasoning about impact and efficiency in the software process, whether AI-assisted or not.