Good but
Needs a bit more stuff, like allowing to edit the system prompt. The rog feature is really good.
bug in iPhone xs
Bug Report: Model Persistence and Startup Errors (iPhone XS)
App Version: Mithril (Local LLM) Device: iPhone XS iOS Version: [ ios 18.3.2]
Issue Summary
I am experiencing two main issues regarding model management and app startup:
Redundant Download Prompt: After successfully importing and downloading a custom model, the app prompts me to download it again every time I select it.
ExecuTorch Error: Every time the app opens, an "ExecuTorch error" appears, even though the model is already present on the device.
Steps to Reproduce
Import a custom model and complete the download process.
Select the model for use.
Observe the prompt asking to download the model again. (Current workaround: I close/force-quit the app to bypass this).
Re-open the app and observe the "ExecuTorch error" notification upon startup.
Expected Behavior
The app should recognize that the model is already stored locally and load it immediately without re-triggering the download prompt.
The app should launch without ExecuTorch errors if the model files are valid and present.
Additional Notes
Despite the error messages, the model does eventually work if I bypass the prompts.
I am using an iPhone XS, so there may be specific optimizations or memory constraints related to older A12 Bionic hardware that are triggering these flags.
App Version: Mithril (Local LLM) Device: iPhone XS iOS Version: [ ios 18.3.2]
Issue Summary
I am experiencing two main issues regarding model management and app startup:
Redundant Download Prompt: After successfully importing and downloading a custom model, the app prompts me to download it again every time I select it.
ExecuTorch Error: Every time the app opens, an "ExecuTorch error" appears, even though the model is already present on the device.
Steps to Reproduce
Import a custom model and complete the download process.
Select the model for use.
Observe the prompt asking to download the model again. (Current workaround: I close/force-quit the app to bypass this).
Re-open the app and observe the "ExecuTorch error" notification upon startup.
Expected Behavior
The app should recognize that the model is already stored locally and load it immediately without re-triggering the download prompt.
The app should launch without ExecuTorch errors if the model files are valid and present.
Additional Notes
Despite the error messages, the model does eventually work if I bypass the prompts.
I am using an iPhone XS, so there may be specific optimizations or memory constraints related to older A12 Bionic hardware that are triggering these flags.
Show more
Works as described so far
I’ve just started using the app and it’s working as described. There is one critical feature missing. It must have a way to quickly copy the generated text. Each request and response needs to be able to be copied easily. Preferably with a button or a shortcut. I will update my rating to 5 stars a fast as possible when this feature is added. If there is a stretch goal for this app I would like to be able to use text2image models as well. Great job, and keep up the great work.
Show more
Best app for local LLMs
Currently the best App Store iOS app for running local LLMs. Despite its many shortcomings it still gets 5 stars simply because there is currently no competition to this app to my knowledge. It is the only app capable of running any model that’s compatible with llama.cpp. This allows the use of custom fine tuned models downloaded from Hugging Face which is essential as 4B parameters is about the maximum that an iPhone can currently handle. It really needs some improvements such as the ability to copy prompts and answers to the system clipboard. Currently the best workaround is to take a screenshot and use the iOS Live Text feature to copy the text to clipboard although this is impractical for long answers.
Some other issues include the export data feature. The exported .json file is terribly formatted.
The app has limited support for markdown and formatting. Failing to render tables for example.
Thinking models also have their output truncated making it impossible to see the models entire process.
Browsing conversation history is annoying. It lacks search feature and is limited to past 100 conversations which is exasperated by the apps poor quality .json exports.
A previous app I have used called “Locally AI” has a very good UX, but has tons of technical limitations.
Some other issues include the export data feature. The exported .json file is terribly formatted.
The app has limited support for markdown and formatting. Failing to render tables for example.
Thinking models also have their output truncated making it impossible to see the models entire process.
Browsing conversation history is annoying. It lacks search feature and is limited to past 100 conversations which is exasperated by the apps poor quality .json exports.
A previous app I have used called “Locally AI” has a very good UX, but has tons of technical limitations.
Show more
Best app for local LLMs
Currently the best App Store iOS app for running local LLMs. Despite its many shortcomings it still gets 5 stars simply because there is currently no competition to this app to my knowledge. It is the only app capable of running any model that’s compatible with llama.cpp. This allows the use of custom fine tuned models downloaded from Hugging Face which is essential as 4B parameters is about the maximum that an iPhone can currently handle. It really needs some improvements such as the ability to copy prompts and answers to the system clipboard. Currently the best workaround for anyone interested is to export data (at the bottom of the screen) and parse the export file, Or take a screenshot and use the iOS Live Text feature to copy the text to clipboard although this is impractical for long answers.
Show more
Perfect for local models
This works better then I would expect on a phone. Would love to be able to host an openai style service on it with it.
Clunky, impossible to paste API
The app is clunky with no instructions. It doesn't allow API key to be pasted. Impossible to contact developer.
