Semiconductor Engineering

Characterization of GPU-based Inference for Reasoning-Centric LLMs (Micron, Argonne)

Researchers from Micron Technology and Argonne National Laboratory have released “Understanding Inference Scaling for LLMs: ...
InfoQ

Cactus v1: Cross-Platform LLM Inference on Mobile with Zero Latency and Full Privacy

A monthly overview of things you need to know as an architect or aspiring architect. Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with ...
Crypto Briefing

MIT’s MeMo framework boosts LLM performance by 26% without retraining

MIT's MeMo framework trains a compact memory model that boosts LLM performance by up to 26.73% without retraining, with major implications for crypto AI agents.
VentureBeat

Researchers baked 3x inference speedups directly into LLM weights — without speculative decoding

As agentic AI workflows multiply the cost and latency of long reasoning chains, a team from the University of Maryland, Lawrence Livermore National Labs, Columbia University and TogetherAI has found a ...
Semiconductor Engineering

Transformers At The Edge: Efficient LLM Deployment

Since the groundbreaking 2017 publication of “Attention Is All You Need,” the transformer architecture has fundamentally reshaped artificial intelligence research and development. This innovation laid ...
SDxCentral

AI inference crisis: Google engineers on why network latency and memory trump compute

Google researchers have warned that large language model (LLM) inference is hitting a wall amid fundamental problems with memory and networking problems, not compute. In a paper authored by ...
techtimes

AI at the Edge: LLM on NVIDIA Jetson

Demand for AI solutions is rising—and with it, the need for edge AI is growing as well, emerging as a key focus in applied machine learning. The launch of LLM on NVIDIA Jetson has become a true ...