A monthly overview of things you need to know as an architect or aspiring architect. Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with ...

Using these new TensorRT-LLM optimizations, NVIDIA has pulled out a huge 2.4x performance leap with its current H100 AI GPU in MLPerf Inference 3.1 to 4.0 with GPT-J tests using an offline scenario.

A new technical paper titled “Pushing the Envelope of LLM Inference on AI-PC and Intel GPUs” was published by researcher at Intel. “The advent of ultra-low-bit LLM models (1/1.58/2-bit), which match ...

The CEOs of OpenAI, Anthropic, and xAI share a strikingly similar vision — AI’s progress is exponential, it will change humanity, and its impact will be greater than most people expect. This is more ...

Use left and right arrow keys to seek audio. Dell has just unleashed its new PowerEdge XE9712 with NVIDIA GB200 NVL72 AI servers, with 30x faster real-time LLM performance over the H100 AI GPU. Dell ...

Xiaomi is reportedly in the process of constructing a massive GPU cluster to significantly invest in artificial intelligence (AI) large language models (LLMs). According to a source cited by Jiemian ...

Companies investing in generative AI find that testing and quality assurance are two of the most critical areas for improvement. Here are four strategies for testing LLMs embedded in generative AI ...

Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources. Dany Lepage discusses the architectural ...

Forward-looking: At Nvidia's GTC 2024 conference on Monday, the company unveiled its Blackwell GPU platform that it says is designed for generative AI processing. The next-gen lineup includes the B200 ...

We've all seen the news reports, stories, and videos surrounding AI development and large language models (LLMs) hosting with huge encompassing data centers. These things can draw similar amounts of ...