I am building a notes application that automatically finds related notes - thinkdeli.com. I want to implement end-to-end encryption sometime soon.
To find related notes, I create embeddings, add them to an index, and find the nearest neighbors.
I am creating embeddings of the user's notes locally using transformers.js. But setting up the pipeline takes up a lot of memory (around 400+ MB on Chrome). This makes it somewhat impractical to use on older devices.
Is there a more efficient way to create embeddings locally?
Creating embeddings via an API will be more efficient, but that would mean sending users unencrypted notes over the cloud to the service. Edit - What I mean is this. The user's notes must be unencrypted and readable as plain text on the server to create embeddings. This defeats the purpose of end-to-end encryption.
I would appreciate any pointers. Thanks a lot!