Can we use Petals on bolt.diy

General
1

I’m looking for ways to run powerful language models without needing a high-end GPU, and I recently discovered Petals. It seems like a fantastic solution because it uses distributed computing, similar to BitTorrent, which makes it possible to run models like Code Llama even on less powerful hardware.

Has anyone here successfully integrated Petals with Bolt.diy? If so, I would love to hear about your experience and any tips or steps you can share to help set it up.

1 Like
2

Never heard of it but what I see looks indeed very interesting

https://petals.dev/

Edit: Just reading the website and saw this:

image
image884×348 21.9 KB

I think thats way to low to use it for bolt. I am not sure but I think it would need at least 40 tokens/s to be smooth and useable, unless you want to wait hours to get your app :smiley:

3 Likes