Every model is free up to 100k monthly active devices. No tokens, no logins.

I dunno about the business model though. Cloud LLM billing makes sense: you're getting another computer to do work with each request, and using their compute via their gateway that they bill you.

These local models are like old school software. They're producing the weights, and then giving them to people. If I'm happy with the weights you've given me, and I'm not using your compute for inference and not wanting or needing any updates off you, why should you continue getting money off me and my customers?

The whole "but we need to keep it updated for your security" doesn't really work as well for software designed to run fully offline like these local models are.

I'm not saying they shouldn't get paid, but I guess I feel a personal sadness that it's less obvious how to successfully monetise such a sincerely useful and beneficial approach towards AI models.

The world ships more than a billion capable phones, tablets, and laptops a year, most with a chip built for exactly this work, paid for and idle most of the day. Run the model there and the economics flip: no per-call cost, no round-trip, and nothing leaves the device.

This. I run small models (>50MB) for bio-imaging/biotech applications, it feels like every README implies that you need a discrete GPU to get started. While some do, many, especially the most useful ones, do not. Sure it matters if you're also going to do fine-tuning, but I believe your typical user just wants to detect some nuclei and get some cell-body ratios.

The laptop on your desk won't be running Meta's SAM, but it has more than enough compute to crunch 100's of your H&E slides overnight.

accessible via one SDK for Swift, Kotlin, and JavaScript

Lol well let me know when there's a Python SDK and I'll give it a try then. Obviously this isn't a deal breaker if you have a real case, but as someone who is willing to spin something up and try it out if there's a quick "pip install" command, this is getting put back on the shelf for now.

I tried out their demo for Clear, the audio quality improvement model.[0] I'm not sure if it's just I don't have refined enough an ear or their demo is broken, but the "raw" and "enhanced" versions sounded exactly the same to me.

[0] https://desertant.com/models/clear/

If any of you are interested there are apps like https://github.com/altic-dev/FluidVoice (this one's a great app) and this https://sam-pop.github.io/WhisperDictation. The latter, even though it has just 7 stars right now, seems to be more "intuitive". I just hope they expose a way to "connect" to available models on the machine or remotely)

Most organisations give their users terrible hardware, so anything which requires 32GB of RAM or a MacBook Pro won’t work if it’s a government or large organisation.

Voz is Parakeet 0.6B v3

Clear is DeepFilterNet 3

Ear is the language predictor from whisper-tiny

...

Ranks a transcript's best non-overlapping moments: each clip gets scored and ranked. Build strong selections or unique editing features to pick the best sentences in video or audio recordings.

This is very impressive for a 248MB model. I wonder how good the results are, as an LLM 10x the size is still quite bad at that.

opinionated on-device intelligence

Hate speech triage. On-device moderation that flags hateful, abusive and threatening text

What could go wrong here?