Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://github.com/qusaismael/localllm below:

qusaismael/localllm: Your LLM, Your Data , Your GUI.

A simple Flask-based web GUI that enables local AI (LLMs) inference using ollama for model serving. This project is currently in Alpha phase and open to any contributions. Created by @qusaismael.

If you'd like to support this project, consider donating via PayPal:

• Features
• System Requirements & Recommendations
• Installation
• Usage
• Security Notice
• Troubleshooting
• Project Status
• Roadmap
• Contributing
• License
• Acknowledgments
• References

• Multiple Model Support: Easily switch between different local LLM models (e.g., deepseek-r1, qwen2.5, codellama, etc.)
• Streaming Responses: See tokens appear in real time using server-sent events (SSE)
• Markdown and Code Block Rendering: Code blocks with syntax highlighting and copy-to-clipboard
• Raw Output Toggle: Debug with raw text output visibility
• Cross-Platform: Works on Windows, Linux, and macOS
• Keyboard Shortcuts:
  • Shift+Enter: New line
  • Enter: Send message

• Python 3.7+
  Required for Flask compatibility

• pip/venv
  For dependency management and environment isolation

• ollama
  Installation required
  Verify installation:

• Hardware:

  • Minimum: 8GB RAM (for smaller models)
  • Recommended: 16GB+ RAM + NVIDIA GPU (for larger models)
  • Disk Space: 10GB+ for model storage

1. Clone Repository

   git clone https://github.com/qusaismael/localllm.git
   cd localllm

2. Setup Virtual Environment

   # Linux/macOS
   python3 -m venv venv
   source venv/bin/activate
   
   # Windows
   python -m venv venv
   venv\Scripts\activate

3. Install Dependencies

   pip install -r requirements.txt

4. Configure Ollama

   • Ensure Ollama is running:
   • Download models first:

     ollama pull deepseek-r1:14b

1. Start Server

   Access at http://localhost:5000

2. First-Time Setup

   • Select model from available options
   • If models aren't listed, ensure they're downloaded via Ollama

3. Basic Operations

   • Type prompt & press Enter to send
   • Toggle raw output for debugging
   • Copy code blocks with one click

⚠️ Important Security Considerations:

• Default binding: 0.0.0.0 (accessible on your network)
• Not recommended for public internet exposure
• No authentication layer implemented
• Use firewall rules to restrict access if needed

Common Issues:

1. "Model not found" error

2. Port conflict Modify PORT variable in app.py

3. Slow responses

   • Try smaller models first
   • Check system resource usage
   • Ensure GPU acceleration is enabled if available

4. Windows path issues Update OLLAMA_PATH in app.py to your installation path

Alpha Release
Current version: 0.1.0

Known Limitations:

• No conversation history
• Basic error handling
• Limited model configuration

• Conversation history support
• Model download UI
• Docker support
• System resource monitoring

Welcome! Please follow these steps:

1. Fork repository
2. Create feature branch
3. Submit PR with description

Development Setup:

pip install -r requirements-dev.txt
pre-commit install

Guidelines:

• Follow PEP8 style
• Add tests for new features
• Update documentation accordingly

MIT License - See LICENSE for details

• Built with Flask
• LLM backend by Ollama
• Inspired by the Open Source AI community

• Ollama Documentation
• Flask Documentation
• CUDA Installation Guide

Created by @qusaismael
Open Source • Contributions Welcome!

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4