NenaMark3 vs MLX Benchmarks

Nenamark3 is a benchmark that measures the 3d performance of a system by gradually increasing the complexity of a scene until the frame rate drops below 60. Purpose Nenamark3 is run to compare different systems against each other from a 3d graphics perspective. A 3d graphics benchmark tries to show how well a system will perform in games or or apps that make use of 3d graphics. 3D Graphics Apps A 3d app, such as a game, will typically have individual settings for different tiers of devices. A lo-end system will have less features and look less amazing than a hi-end system. If hi-end settings are used on a lo-end system it will not be able to maintain its frame rate of 60 fps, causing it to drop frames and “stutter”. What a 3D benchmark should do This means that a graphics benchmark should define a set of scenes of increasing complexity and find how far a system can go while maintaining 60 fps. The common way of running static scenes and measure the fps have several problems. First, it is not relevant to run at any other frame rate than 60. It is not certain that lower fps are comparable to each other. All we can say for certain is - did it make 60 or not? Second, once 60 is reached where do you go? While it is possible to do off screen render and let the fps go as high as it can this is also irrelevant. If you have more power you would probably want more advanced features rather than more of the lesser features (perhaps you’d want a more correct lighting model rather than say more geometry). Go Slow There is a point to scaling the complexity gradually over time. Especially embedded systems have quite sophisticated clock governors that speeds up the clocks when needed, even for the GPU. These sample the load at certain intervals and adapt so it makes sense to slowly build complexity. In a game there would be a few slow frames before the system could react which would be annoying but not critical.

MLX Benchmark helps you understand how large language models perform on your hardware. Select a model and quantization, set context length and iterations, and run repeatable benchmarks that capture load time, tokenization time, time to first token, throughput, memory usage, and jitter. View detailed metrics grids, visualize comparisons in charts, and manage downloaded model repos directly in the app. Built for developers and AI enthusiasts who want clear, on-device performance insights without leaving their device. Features: - Model selection with quantization details - Configurable prompt, context length, and iterations - Metrics for load, tokenization, first-token latency, throughput, jitter, and memory - Charts to compare runs and devices - Downloaded model management (list, delete, refresh) - Optional Supabase upload for aggregating benchmark results