LFM3 1.7b - Rather good

I finally got around to trying it. This tiny model is surprisingly capable it seems. Also tried the 2b VL - also very good but it has half the context of 1.7b. It's also quite fast: $ llama-bench -m LFM3-1.7B-Q8_0.gguf -fa 1 ggml_cuda_init: found 1 CUDA devices (Total VRAM: 24112 MiB): Device 0: NVIDIA GeForce RTX 3090 Ti, compute capability 8.6, VMM: yes, VRAM: 24112 MiB | model | size | params | backend | ngl | fa | test | t/s | | ------------------------------ | ---------: | ---------: | ---------- | --: | --: | --------------: | -------------------: | | lfm3 1.7B Q8_0 | 1.71 GiB | 1.73 B | CUDA | -1 | 1 | pp512 | 20648.61 ± 1786.65 | | lfm3 1.7B Q8_0 | 1.71 GiB | 1.73 B | CUDA | -1 | 1 | tg128 | 329.03 ± 2.05 | build: 28b812d6c (12087) This makes me want to try their omni model for watching videos. Is there a simple way to get this working? Ideally, I'd like to throw a video file in the chat or give it a youtube link.

参与讨论