Benchmarks
Design generation
-
The Figma prototype: I build my ideas #8 - 07/19/20
{
"navigation": [
{"icon": "camera"},
{"title: "Photos"},
{"icon": "message"}
],
"photos": [
{"icon": "user"},
{"frame": "photo"},
{"icon": "heart"},
{"icon": "chat bubble"}
]
}
Code generation
- Builder codegen
- React Tailwind code:
https://twitter.com/gabe_ragland/status/1598068207994429441
- https://www.patterns.app/blog/2023/01/18/crunchbot-sql-analyst-gpt/ “given this database .. generate query to answer .. now check for these mistakes. Now fix query because db returned error .. now loop”
- https://twitter.com/thekitze/status/1639724609112096768?s=20
Interactive codebase tools
- https://github.com/BloopAI/bloop: Code search and explanation
- Copilot X
- https://www.cursor.so/: IDE, with next edit prediction
- https://www.codium.ai/: test generation, code search
- Cody from Sourcegraph (OSS): like an open source Copilot
- https://github.com/CodedotAl/gpt-code-clippy: abandoned
- https://github.com/mpoon/gpt-repository-loader: just concat
- refact.ai: completion, refactoring, chat
- https://reviewify.io/
- https://continue.dev/ (Apache): somewhat Cody like
- LocalPilot: just routes Copilot requests to a local LLM instead of Github; can use mitmproxy instead (HN)
- Codeium
- https://github.com/huggingface/llm-vscode completions
Coding agents
- aiden: edit existing codebases
- ‣ (MIT): generate from scratch. uses tools?
- https://github.com/AntonOsika/gpt-engineer (MIT): generate from scratch
- https://github.com/emrgnt-cmplxty/automata (Apache)
- smol https://github.com/smol-ai/developer(MIT)
- https://github.com/sweepai/sweep (Elastic): edit existing codebases
- https://github.com/joshpxyne/gpt-migrate (MIT)
- Codegen.com
- https://blog.research.google/2023/05/large-sequence-models-for-software.html?m=1
- https://blog.research.google/2022/07/ml-enhanced-code-completion-improves.html?m=1
Code interpreter
Easy end-user inference: llama.cpp, ollama, gpt4all