b8658
server: save and clear idle slots on new task ( --clear-idle ) ( #20993 ) server: clear idle slots KV from VRAM (LLAMA_KV_KEEP_ONLY_ACTIVE) server: move idle slot KV clearing to slot release The save "cost" is now paid by the finishing request. server: add --kv-clear-idle flag, enable by default server: skip clearing last idle slot, clear on launch server: test --no-kv-clear-idle flag server: simplify on-release clearing loop server: remove on-release KV clearing, keep launch-only cont : clean-up tests: update log strings after --clear-idle rename tests: use debug tags instead of log message matching test: fix Windows CI by dropping temp log file unlink Co-authored-by: Georgi Gerganov [email protected] macOS/iOS: macOS Apple Silicon (arm64) macOS Intel (x64) iOS XCFramework Linux: Ubuntu
server: save and clear idle slots on new task (--clear-idle) (#20993)
-
server: clear idle slots KV from VRAM (LLAMA_KV_KEEP_ONLY_ACTIVE)
-
server: move idle slot KV clearing to slot release
The save "cost" is now paid by the finishing request.
-
server: add --kv-clear-idle flag, enable by default
-
server: skip clearing last idle slot, clear on launch
-
server: test --no-kv-clear-idle flag
-
server: simplify on-release clearing loop
-
server: remove on-release KV clearing, keep launch-only
-
cont : clean-up
-
tests: update log strings after --clear-idle rename
-
tests: use debug tags instead of log message matching
-
test: fix Windows CI by dropping temp log file unlink
Co-authored-by: Georgi Gerganov [email protected]
macOS/iOS:
-
macOS Apple Silicon (arm64)
-
macOS Intel (x64)
-
iOS XCFramework
Linux:
-
Ubuntu x64 (CPU)
-
Ubuntu arm64 (CPU)
-
Ubuntu s390x (CPU)
-
Ubuntu x64 (Vulkan)
-
Ubuntu arm64 (Vulkan)
-
Ubuntu x64 (ROCm 7.2)
-
Ubuntu x64 (OpenVINO)
Windows:
-
Windows x64 (CPU)
-
Windows arm64 (CPU)
-
Windows x64 (CUDA 12) - CUDA 12.4 DLLs
-
Windows x64 (CUDA 13) - CUDA 13.1 DLLs
-
Windows x64 (Vulkan)
-
Windows x64 (SYCL)
-
Windows x64 (HIP)
openEuler:
-
openEuler x86 (310p)
-
openEuler x86 (910b, ACL Graph)
-
openEuler aarch64 (310p)
-
openEuler aarch64 (910b, ACL Graph)
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
llamareleaselaunch
AI Is Insatiable
While browsing our website a few weeks ago, I stumbled upon “ How and When the Memory Chip Shortage Will End ” by Senior Editor Samuel K. Moore. His analysis focuses on the current DRAM shortage caused by AI hyperscalers’ ravenous appetite for memory, a major constraint on the speed at which large language models run. Moore provides a clear explanation of the shortage, particularly for high bandwidth memory (HBM). As we and the rest of the tech media have documented, AI is a resource hog. AI electricity consumption could account for up to 12 percent of all U.S. power by 2028. Generative AI queries consumed 15 terawatt-hours in 2025 and are projected to consume 347 TWh by 2030. Water consumption for cooling AI data centers is predicted to double or even quadruple by 2028 compared to 2023. B
OpenAI Releases Policy Recommendations for AI Age
OpenAI has released policy recommendations to address the rapid social changes driven by AI. OpenAI's Chief Global Affairs Officer Chris Lehane discusses the company’s ideas to “ensure AI benefits everyone.” Lehane joins Caroline Hyde and Ed Ludlow on “Bloomberg Tech.” (Source: Bloomberg)
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
More in Releases
OpenAI Releases Policy Recommendations for AI Age
OpenAI has released policy recommendations to address the rapid social changes driven by AI. OpenAI's Chief Global Affairs Officer Chris Lehane discusses the company’s ideas to “ensure AI benefits everyone.” Lehane joins Caroline Hyde and Ed Ludlow on “Bloomberg Tech.” (Source: Bloomberg)
![[OpenAI] Industrial policy for the Intelligence Age](https://d2xsxph8kpxj0f.cloudfront.net/310419663032563854/konzwo8nGf8Z4uZsMefwMr/default-img-matrix-rain-CvjLrWJiXfamUnvj5xT9J9.webp)
[OpenAI] Industrial policy for the Intelligence Age
As we move toward superintelligence, incremental policy updates won’t be enough. To kick-start this much needed conversation, OpenAI is offering a slate of people-first policy ideas(opens in a new window) designed to expand opportunity, share prosperity, and build resilient institutions—ensuring that advanced AI benefits everyone. These ideas are ambitious, but intentionally early and exploratory. We offer them not as a comprehensive or final set of recommendations, but as a starting point for discussion that we invite others to build on, refine, challenge, or choose among through the democratic process. To help sustain momentum, OpenAI is: welcoming and organizing feedback through [email protected] establishing a pilot program of fellowships and focused research grants of u
AIs can now often do massive easy-to-verify SWE tasks and I've updated towards shorter timelines
I've recently updated towards substantially shorter AI timelines and much faster progress in some areas. [1] The largest updates I've made are (1) an almost 2x higher probability of full AI R&D automation by EOY 2028 (I'm now a bit below 30% [2] while I was previously expecting around 15% ; my guesses are pretty reflectively unstable) and (2) I expect much stronger short-term performance on massive and pretty difficult but easy-and-cheap-to-verify software engineering (SWE) tasks that don't require that much novel ideation [3] . For instance, I expect that by EOY 2026, AIs will have a 50%-reliability [4] time horizon of years to decades on reasonably difficult easy-and-cheap-to-verify SWE tasks that don't require much ideation (while the high reliability—for instance, 90%—time horizon will
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!