MLPerf Mobile vs Pipette by Liquid

MLPerf Mobile is a free, open-source benchmarking tool designed to measure the performance of mobile devices like smartphones and tablets in various artificial intelligence (AI) and machine learning (ML) tasks. Workloads tested include image classification, language understanding, super resolution upscaling, and image generation based on text prompts. This benchmark makes use of hardware AI acceleration on many of the latest mobile devices to ensure the best performance where possible.

How fast can your phone run real AI? Pipette is an on-device AI benchmark app. It runs small models on your phone and measures actual performance without cloud processing. See how quickly your device processes prompts and generates text, measured in tokens per second. Many mobile benchmarks estimate AI performance using synthetic tests. Pipette downloads real, openly available AI models and runs them locally with the inference engine built into the app. The result is a benchmark that reflects how an offline AI assistant, local chatbot, or other on-device AI application would actually perform on your phone. What you can do with Pipette • Run AI models on your phone • Benchmark prefill speed, generation speed, latency, and peak memory use • Test popular model families like Liquid, Qwen, Gemma, Llama, Granite, and Mistral • Compare model sizes and quantization levels such as Q4 and Q5 • See results in tokens per second with per-model breakdowns • Use thermal readiness checks for more consistent runs • Export benchmark results as CSV • Optionally contribute results to Liquid AI's public dataset • And more… Fair, repeatable results Device temperature can have a major effect on performance. Pipette includes a thermal readiness check that waits until your phone is cool enough before starting a benchmark. This helps produce more consistent results across devices and repeated runs. Who Pipette is for • Developers building local AI, offline AI, and on-device AI apps • Hardware enthusiasts comparing smartphones and chipsets • Reviewers who need a repeatable mobile AI benchmark • Researchers and community members who want to contribute to Liquid AI's benchmark database • Anyone curious about how well their phone can run AI locally How it works • Sign in with your email • Choose a benchmark and the context length • Download the AI model • Run the test and view your results AI model downloads may be several gigabytes. A Wi-Fi connection is recommended. Pipette is built by Liquid AI, creators of Liquid Foundation Models, efficient AI models designed for on-device inference.