Local LLM Server vs Sidecar ML

Transform your iPhone, iPad, or Mac into a private, local LLM AI server. This powerful utility allows you to run Large Language Models (LLMs) powered by Apple Intelligence directly on your device, providing an OpenAI-compatible API and Ollama compatible API for your development and testing needs. Run a local AI LLM server on your Apple Intelligence enabled device to give private, local edge AI abilities to your applications. Note: This app requires iOS 26.0/macOS 26.0 or above to run Apple Intelligence models. Free version: 100 free requests per day, resetting at midnight. Local LLM Server Pro (separate paid app) has no limit, no ads, and no tracking. **Key Features:** * **Local Server Hosting:** Run a fully functional LLM server right from your pocket. No internet connection is required. * **OpenAI-Compatible API:** Seamlessly integrate with existing tools and clients that support standard chat completion endpoints (OpenAI and Ollama API). * **Real-Time Monitoring:** Keep track of your server's information with a live status dashboard showing IP address, active port, and logs. * **Live Logs:** View real-time server logs to debug requests and monitor performance instantly. * **Developer Friendly:** Includes built-in curl command example to quickly test your server. Whether you are an AI researcher, a developer building private apps, or an enthusiast exploring local intelligence, this app provides the local LLM AI infrastructure you need using the devices you already have. This app is not affiliated with nor is endorsed by Apple. Use responsibly and in accordance with Apple's terms of service.

Turn your iPhone into a private, on-device AI server for your whole network. Sidecar ML runs a lightweight HTTP server on your iPhone that exposes Apple's built-in machine-learning models to any device on your local network. Tap Start Server, scan the QR code, and your Mac, PC, Raspberry Pi, or browser can call powerful on-device AI over plain REST — no cloud, no accounts, no API keys, no usage fees, and no data leaving your network. Everything runs on the phone. Nothing is uploaded to a server and there's nothing to pay for. (The one exception, music recognition, matches against Apple's catalog and tells you so up front.) ONE IPHONE, A DOZEN AI CAPABILITIES • Chat with the on-device large language model through a chat-completions endpoint that's compatible with the popular OpenAI API format — streaming, JSON mode, and structured output included. Point existing tools and SDKs at your phone and they just work. • Generate images from a text prompt in a choice of art styles. • OCR — pull text out of photos, receipts, and screenshots with per-line boxes and multi-language support. • Read barcodes and QR codes (EAN, Code128, and more). • Classify images across roughly a thousand categories, or produce image embeddings for similarity search. • Remove backgrounds and isolate subjects or people as clean PNG cut-outs. • Detect faces and facial landmarks, body pose, and hand pose. • Scan documents with automatic edge detection and perspective correction. • Convert speech to text and text to speech, with a wide range of voices (including Personal Voice). • Translate text between languages completely offline. • Analyse text: language, sentiment, named entities, tokens, and embeddings. • Classify sounds across hundreds of categories, or identify the music that's playing. BUILT FOR DEVELOPERS AND MAKERS • Clean REST + Server-Sent Events API — call it from curl, Python, JavaScript, or any HTTP client. • Chat endpoint compatible with the OpenAI format, so many existing tools and SDKs work unchanged. • Automatic discovery on your network via Bonjour. • Honest capability reporting: ask the server what it can do and it tells you exactly what's available on your device, and why. • Optional bearer-token authentication to lock the server down on shared networks. FREE WEB CONSOLE AND CODE EXAMPLES A ready-to-use web console — plus Python examples, a command-line client, and full API documentation — is free and open-source. Download the single-file console and browse the source here: https://github.com/AndersFB/Sidecar-ML-Console/ Open the console in any browser, enter your phone's address, and try every capability with a friendly UI — nothing to install. GOOD TO KNOW • The on-device large language model and image generation require an Apple Intelligence-capable iPhone. The other capabilities run on a much wider range of devices. • The server runs while the app is in the foreground; a keep-awake option helps during longer sessions. • Some features (offline translation language pairs and certain speech models) download their models on first use. Turn the iPhone in your pocket into a private AI endpoint for your projects, your home lab, and your prototypes — Sidecar ML.