Leave a Review Private LLM
So far so good!
Okay… THIS APP IS AMAZING. I am using this on my iPhone 12 Pro Max and wow, pretty fast reply. I am still playing around with this app but I am very impressed with it. I wonder if new models from huggingface could be downloaded, if that’s possible.Junayet
Developer Response
Thanks for the review! We'll be adding support for downloading multiple models soon. This will be released on the macOS version of the app first (please try it, if you haven't done so already, it's a free download on the Mac App Store, since you've already got the iOS version of the app, and also, it supports bigger and better models). Currently, it isn't possible to support downloading arbitrary models from huggingface because our app uses a compilation based approach for performance and support for each model has to be baked into the app binary.So much potential
I've been using this app for a couple of months at this point. When it was first released, it was a neat proof of concept, but it only supported 7B models and was overall just too simple to use effectively. With a recent update, a 13B model was released for all macs with 16GB of memory, and it makes such a huge difference! It's not quite at the same level as ChatGPT 3.5T, but it's close enough that I never use 3.5T anymore; this is my go-to. I greatly appreciate the on-device processing (hallelujah privacy!), and it doesn't even use too much power - my battery still lasts for hours and hours. The performance is also great; my base M1 Air powers right through the prompts.
I only have two complaints about the app at this point. 1) The 13B model uses about 12GB of memory by itself, which does force the use of swap on a 16GB Air. Not much the dev can do about this, but it is something to keep in mind. You'll want to close out of other programs before launching this. 2) There still is no feature that has separate conversations. If you want to start a new conversation, you need to delete the existing conversation. I'd love it if we could get separate conversations in a future update; it would make this app so much easier to use.
Overall, I love it and do not regret buying it at all. I can't wait to see what future updates bring :)Show lessN7nathan
Great app.
Would love to be able to download different models. If not from an in app catalogue, perhaps putting in a url from hugging face?
Otherwise pretty neat.dtlnx
7b crashes, 3b is trash
7b crashes on iPhone 14 pro. 3b isn’t worth using, can’t list the presidents.Funnyfinn13
Developer Response
We'd received one more bug report for this issue. We've released v1.6.2 today, to fix this issue. Although, the model weights have changed, if you've already downloaded the 7B model, you might have to delete and reinstall the app. We'll soon add a UI within the app for users to be able to download and manage additional 7B models.Works well but needs some improvement
The 7B model constantly crashes on 14 Pro even though the prompt says that the phone can handle it. Downgrading to the 3B seems to work much better.jingumistrad33246
Developer Response
Thanks for the review and the bug report. We've just released v1.6.2 of the app that fixes this issue. Although, the model weights have changed, if you've already downloaded the 7B model, you might have to delete and reinstall the app. We'll soon add UI within the app for users to be able to download and manage additional 7B models.Please allow LLM choice
Please allow us to pick our LLM the same way we do in text-generation-ui by providing hugging face name. I realize not all models can run on iOS, but I was hoping to mostly use this for CodeLlama.Triplet97
Developer Response
Thanks for the review! Memory is the primary bottleneck for supporting larger models on iOS. Fortunately, quantization tech is steadily getting better. We plan on adding support for more 7B models on newer iOS devices (iPhone 14+), soon. Specifically for CodeLlama 7B, supporting the full 16k context length of the model on iOS devices with 6-8GB of RAM is going to be quite challenging, but we could support it with reduced context length. Please try our macOS app (free download since you've already got the iOS version) which supports 13B models on Apple Silicon Macs with 16GB or more RAM. We're also planning on supporting even bigger models on Apple Silicon Macs with more RAM.Fantastic
Hard to believe this kind of power is available locally. Can’t wait to see how this tech develops. Would love to be able to upload documents or large text files.ben_hdk853
no correct answers on language
asked it english word meanings and it failedS Trq
Developer Response
Could you please elaborate? We've never received any negative feedback on our app's English abilities. What word(s) did it fail on? Also was this with the default 7B model or the downloadable 13B model? Please email us: support@numen.ie or DM us on X: @private_llm and we'll make it right. Edit: 10th Oct 2023, we haven't heard back from the reviewer, leading us to conclude that this review wasn't posted in good faith.wish there will be a local api compatible to openai, and different chatboxes
great appetty__
This is a sweet personal AI, excellent work!!!
Thank you for this personal AI. So far it has not gone astray, and has given fairly correct answers and did so very quickly on an iPhone 13 Pro.
I’m very happy.
I would like to load additional data from several hundred research PDF’s that I have, and upload forensic graphic content. If it can do that my world will be forever changed! 👍👍👍
Thank you again for such a wonderful Personal GPT!Show lessJoe Rollins
