Serguey Asael Shinder: A speedup is a claim about your workload, not about the library
"Six to eighteen times faster" is the kind of headline that invites two wrong responses: believe it unconditionally, or dismiss it as benchmarking theatre. Both skip the only question that matters, which is faster at what, for whom. A speedup is never a property of a library in the abstract. It is a property of a specific operation, on a specific distribution of inputs, measured against a specific baseline — and whether it reaches your code depends entirely on how much of your time is spent in the thing that got faster.
What separates a useful performance claim from an empty one is whether it hands you the denominator. The JDK 28 change to off-heap allocation is a good example to think with, precisely because its write-up does not stop at the multiplier. It says the win applies to small allocations — under 64 bytes — made in confined arenas, the scratch-space pattern around native calls, and it backs that with a measured distribution: in one instrumented run, over 99.99% of confined arenas that allocated anything stayed under 64 bytes. That is not decoration. It is the information you need to predict your own result. If your application opens confined arenas for tiny native scratch space in a hot loop, the number is probably real for you. If you allocate megabyte buffers, or rarely touch the FFM API at all, the same change does approximately nothing, and the honest version of the announcement tells you so by naming the case it does not cover.
This is the habit worth building when you read any "N times faster": go looking for the workload the number was measured on, and compare it to yours before you let it change a decision. A speedup with a workload attached is a testable prediction you can hold the library to. A speedup with no workload is a mood. The difference is not whether the benchmark is honest — most are — but whether it told you enough to know if it is about you. When it does not, the right move is not to argue with the number; it is to measure your own case, because the only benchmark that can authorise a change to your code is the one that runs your code.