Leave a Review Private LLM
Not worth the money
Quite simply the app is nowhere near useful nor polished enough to warrant the price. For some more specific feedback to the developer:
- there have been better performing low parameter models available for some time, why are these not used?
- unless you are using sane defaults, at least make other models available
- the UI for downloading larger models is poorly thought out, surely you can think of better ways of presenting this?
- history would have been simple enough to implement
Running these things locally is certainly desirable, and feasible, but it seems we will have to wait some time until it’s packaged nicely.Show lessGoblinPlonk
Developer Response
Thanks for your feedback! It looks like this review was written based on v1.6.7 of the app. We'd released v1.7.0 update a few days ago which addresses all the issues you've mentioned except for conversation history, which will come soon. I'd really appreciate it if you could try the updated app and let us know if it's any better now.Gemma 2B just made this a killer app.
With the recent addition of Gemma 2B which is based of Google Gemni this has just become a killer app for me this has been the best LLM model that provides the best answer I hope to see 7B support in the future and I hope this comes Gemma comes to iPhone and iPad soon.
Not only is ther the best performing model but it's all done on device which make this even better. Thank you!Show lessUniqueAirStream
Developer Response
Thanks for the review! We've released Gemma 2B for iPhone and iPads in a recent update.App very buggy but has potential
Only load their custom model. Struggles to download 7B I only give this app 3 stars for its shortcuts integration, pretty neat idea to string custom LLM to Siri the rest of the app is lacking badly and isn’t being maintained or updated don’t waste your money I’ll be requesting a refund may repurchase if updated more like LLM farmShow lesshackerlocal.us
Developer Response
Thanks for the review! Unlike LLM Farm, our app isn't based on llama.cpp. The downside of this is that you cannot download arbitrary models as GGUF files, but the upside of it is better performance because the inference is specialized and compiled for each model and support for cutting edge features like sliding window attention in Mistral models and attention sinks, which llama.cpp and apps that wrap it, lack. We'd been focused on the macOS version of the app until recently, it supports many different models families and just released support for Mixtral-8x7B on it. Please try it, it's a free download for anyone who has the iOS version of the app. We're working on a big update to the iOS app to support many more downloadable models, which should be out in the next two weeks. Also, please feel free to email us, or reach out to us on Discord or X if you have any suggestions or requests for models.Good try
Well, according to the list of viable machines, my iPad Air 3rd generation should be able to run this. It won't run. It launches, struggles for a minute, and crashes. So, I can't really review this. All I can say is that it won't run on my iPad. EDIT: I was able to run this on my MacBook M2 just fine. Looking forward to seeing it run on my iPad. (I have poor old-person eyesight, and I hope the type is slightly larger on the iPad. It doesn't seem to be adjustable on the Mac.)Show lessgitchel
Developer Response
Thanks for the review! The 3rd Gen iPad Air had 3GB of RAM of which just under 2GB is usable for apps, which is just barely enough to load the current quantized 3B model. If you have any other apps running in the background, the app likely won't have enough memory to load the LLM. We're working on an update to the app which will add a new, smaller (but better) default LLM that will work well on your iPad Air. We're internally testing this update on an iPad Air 3rd Gen and an iPhone SE 2 (which also has 3GB of RAM). Edit: The v1.7.0 update is out, which introduces three new models that will work well on your iPad Air 3rd Gen, could you please try the updated version of the app?Privacy focused
After I saw the only review here on the App Store that’s nothing but gibberish, I had to write my own. I love that this app works offline and everything you put into it is private and kept on your device. Plus, there’s no money hungry subscription service. From what I’ve seen on reddit, the developer is working hard on improving the app and I’m looking forward to seeing what’s implemented in the coming updates. So far it’s been great for every day questions and information.Show lessWiredHeart86
Don’t waste your money
Although the software calls itself, an LLM, it’s pretty useless since it hallucinates, and provides a ton of inaccurate information.
It has no access to real time information, even when connected to the Internet. it pretends to have capabilities (booking calendar entries), but it can’t do any of it. not sure where somebody would find the suffer useful.Show lessedward.k
Developer Response
Firstly, please get a refund. Secondly, you might want to look into what LLMs really are. And finally, please look into what are offline LLMs.just crashes constantly
it worked on the default model until i tried downloading another one. then it crashed and now crashed whenever i try to open it. weirdly i deleted the whole app and tried again, same thing. on an M1 Pro/16gb. I'd like a refund.hiburn8
Developer Response
Thanks for the review. Since we don't have any telemetry in the app, we'd appreciate a few details to debug this issue. Could you tell us what's the model that was crashing, also could you tell us how much free memory you have on your machine (from the Memory tab in Activity monitor)? Please email us or contact us on Twitter or Discord. Edit: Are you on the previous version of macOS (Ventura) by any chance? If yes, the occasional crash issue on it has been fixed in the v1.7.6 update.No updates
Hasn’t updated to the newer modelsvtard
Developer Response
We've been working on an update to the iOS app to support multiple downloadable model families for quite some time now and are pretty close to releasing it in the coming weeks. The large matrix of supported devices and model families makes it quite a challenging update to both implement and test. Please join our discord if you'd like to follow along and would like to try the update before it's released. EDIT: v1.7.0 of the app is out and it lets users download about a dozen new models, also has a new Stable LM 2 1.6B builtin model, which is much better than the older RedPajama 3B Chat builtin model. PTALNice app, but can't seem to activate 7B model
iPhone 15. Was most interested in using the 7B model, downloaded it when prompted, have it checked in the settings, but doesn't appear to use the 7B model no matter how many times I hard close and reopen the app. Have even tried deleting all and reinstalling.Tobz_Smith
Developer Response
Thanks for the review! This issue (7B model not working on iPhone 15 (not Plus, Pro or Pro Max)) is something that our users have reported on our Discord. We've even bought an iPhone 15 device to troubleshoot and fix this. The issue seems to be that the iPhone 15 reports only 2.78GB of free memory while the 7B model needs at least 3.2GB of RAM to load. Interestingly enough the iPhone 14 Pro and Pro Max, which have the exact same CPU and RAM as the iPhone 15 (A16 Bionic and 6GB of RAM), don't seem to have this issue. We're working on an update with some more memory optimizations and lower precision quantization specifically for iPhone 15. Please email us or join our discord if you'd like to correspond with us.Missing features.
We need the ability to set the system message, considering the price we should be able to load our own models.
——
Changed to 5 star for now, if the dev delivers on what they said this app should be great!Johnny5AliveNow
Developer Response
Thanks for the review! We've got an update in the works which has both the ability to set the system prompt and also download multiple models, like in the macOS version of the app. We've got a few stability issues to iron out before we can release this update. Also, please try the macOS version of the app, it support bigger models and is a free download for you, since you've already got the iOS version of the app.
