My LLM Toolbox

This a dump of tools, libraries, and other LLM-related things I have collected.

Note: I probably haven't actually used 75% of the stuff, so I am not endorsing anything in particular

Models

Function Calling

• Gorilla
• Functionary

Voice

Text-to-Speech

• Piper: Fast!
• Coqui TTS
• OpenVoice
• MetaVoice
• Bark: Generates any sound, not just voice!
• WhisperSpeech
• Amphion
• MeloTTS
• VoiceCraft: Speech editing

Voice-to-Text

• Whisper

Leaderboards

• Chatbot
• Function Calling

Application

• AutoGPT (never actually even tried it lol)
• 01: Language Model Computer
• podgenai: AI generated podcasts (both content and voice)

CLI

• gorilla-cli
• aiac: Generate IaC files from Terraform to CI config files
• llm: Prompt models from the CLI
• Open Interpreter

NeoVim

• llm.nvim

Local

• Ollama: Run models locally and easily
• Web LLM: Run models in the browser!
• LocalAI
• OpenDevin
• GPT4All: Local ChatGPT
• Llama
• Llama.cpp
• BionicGPT
• Jan
• Open WebUI

Desktop Integration

• ChatGPT Gnome Extension
• ChatGPT Desktop Client

Developer Tools

Example Apps

• Production RAG
• SEC Insights
• Chat LangChain
• DocsGPT

Frameworks

• LangChain
• LlamaIndex
• AutoGen
• fastRAG

Data Loaders

• Unstructured

Structured Output

• Instructor
• SGLang
• LMQL
• guidance
• GPTScript
• Fructose: Strongly-typed LLM calls
• DSPy: Programming, not prompting, models

Embeddings

• sentence-tranfomers

Vector Databases

• Qdrant
• RediSearch
• Weaviate
• Milvus
• pgvector

Observability

• Langfuse
• Openllmetry

Evaluations (Evals)

• Ragas

Security

• Rebuff: Self-hardening prompt injection detector
• E2B: Sandboxed Python enviroment for AI agents

Packaging

• Llamafile
• Cog: Containers for ML models

Low/No Code

• Flowise
• Langflow
• OpenCopilot

Go

I made the dumb mistake of trying to make a RAG app in Go. The ecosystem is way behind Python and JavaScript, but still found some promising projects.

• LangChain Go
• Cybertron: Use pre-trained transfomers from HuggingFace for tasks like vector embedding, etc.
• RediSearch Client