https://github.com/LUC4S007/flask-redis-file-manager/releases

Flask Redis File Manager: Chunked Uploads with Redis Backend

A fast, reliable web app built with Flask. It lets you upload, save, download, rename, and manage text and binary files using Redis as the storage backend. It supports chunked uploads, user and admin roles, and Docker deployment. This project aims to be simple to run, easy to extend, and ready for production in small to medium workloads.

Emojis help navigate ideas, and shields.io badges add quick context. The project topic areas include admin, backend, chunked-upload, docker, file-management, file-storage, file-upload, flask, python, redis, user-management, and web applications.

Table of contents

• Overview
• Why this project
• Key features
• Tech stack
• Architecture and data model
• Getting started
• Docker deployment
• Deployment variants
• Using the API
• Security and access control
• Performance and scaling
• Testing and quality
• Environment configuration
• Customization and extension
• Operation and maintenance
• Troubleshooting
• Releases and updates
• Credits and license
• Roadmap

Overview This project provides a lightweight backend for handling files and text. It stores data in Redis with a focus on chunked uploads to support unreliable network conditions and large files. It combines a Flask app with a Redis persistence layer. The admin and user roles let you restrict actions like renaming, deleting, or listing files. Docker support makes it simple to run anywhere.

Why this project

• Redis as a storage back end gives you fast in-memory operations with durable persistence when configured properly.
• Chunked uploads reduce memory pressure and improve resilience on flaky networks.
• Role-based access control helps keep admin actions separate from regular user actions.
• Docker support makes deployment repeatable and portable.

Key features

• Upload files and text: supports both small and large payloads.
• Chunked uploads: upload large files in segments, then reassemble on the server.
• Save, rename, download, delete: manage files in a simple, consistent way.
• Admin/User roles: control what different users can do.
• Redis-backed storage: fast operations with a stable persistence layer.
• Docker-ready: run locally or in production with Docker Compose.
• Simple, clear API endpoints for automation and integration.
• Base64 support for safe transport of binary data in JSON payloads.

Tech stack

• Python 3.x
• Flask: lightweight web framework
• Redis: in-memory data store with optional persistence
• Docker: containerized deployment
• Base64 handling: for safe binary data transport
• Optional Nginx reverse proxy in advanced setups
• Web client interoperability: RESTful endpoints for admin and user actions

Architecture and data model

• Flask app layer: routes for file operations (upload, download, list, rename, delete)

• Redis layer: stores file metadata and file chunks

• Chunked upload handler: accepts chunks, stores them with a consistent key, and reassembles on complete uploads

• Access control layer: guards endpoints based on user roles

• Optional front-end or API clients: for automation, scripts, or web dashboards

• Docker layer: containerized deployment and orchestration with docker-compose

• Data model concepts:

  • File metadata: id, name, size, owner, created_at, updated_at, mime_type
  • Chunks: stored with keys like file:{id}:chunk:{index}, plus total_chunks and checksum
  • Redis structure: hashes for metadata, lists or streams for chunk order, and string keys for status
  • Access control: role field in user data; ACL checks guard sensitive operations

• Design notes:

  • Chunking reduces peak memory usage on the server during large uploads.
  • Chunk integrity checks help ensure data correctness across retries.
  • Redis storage makes reads fast; consider Redis persistence or replication for durability.

Getting started Environment prerequisites

• Python 3.9+ (or as specified in requirements)
• Redis server (local or remote)
• Docker and Docker Compose (for containerized deployment)

Local development (without Docker)

1. Create a virtual environment
   • python -m venv venv
   • source venv/bin/activate (unix/macOS) or venv\Scripts\activate (Windows)
2. Install dependencies
   • pip install -r requirements.txt
3. Configure environment
   • Create a .env file or set environment variables:
     • FLASK_APP=app
     • FLASK_ENV=development
     • REDIS_URL=redis://localhost:6379/0
     • APP_SECRET_KEY=your-secret-key
     • ADMIN_USERS=admin@example.com
4. Run the app
   • flask run
5. Test endpoints locally
   • Use curl or a client to exercise upload, download, rename, and delete operations

Docker deployment

• The project ships with a Docker-ready setup to simplify deployment.
• Typical workflow:
  • docker-compose up -d
  • The Redis container provides storage, the Flask container serves the app.
• Example docker-compose.yml (conceptual)
  • version: '3'
  • services:
    • redis: image: redis:7-alpine ports: - "6379:6379" command: ["redis-server", "--appendonly", "yes"]
    • web: build: . ports: - "5000:5000" environment: - REDIS_URL=redis://redis:6379/0 - APP_SECRET_KEY=change_me depends_on: - redis
• Docker image: the project can be built into a container that runs the Flask app and connects to Redis. Follow the repository’s Dockerfile for specifics.

Deployment variants

• Local development with Docker Compose
• Kubernetes deployment (advanced): consider a Deployment for the Flask app and a Redis statefulset, with a Service to expose the API.
• Cloud run options: for small deployments, you can use managed Redis and a container running the Flask app in a container service.

Using the API Core endpoints (conceptual examples)

• Create a user or login (authorization flow will depend on your chosen method)
• Upload a file in chunks
  • POST /upload_chunk
  • Accepts: file_id, chunk_index, total_chunks, chunk_data (binary or base64-encoded)
• Complete upload
  • POST /upload_complete
  • Accepts: file_id, name, mime_type, total_size
• Download a file
  • GET /files/{file_id}
• List files
  • GET /files
• Rename a file
  • POST /files/{file_id}/rename
• Delete a file
  • DELETE /files/{file_id}
• Admin-only operations
  • POST /admin/users
  • POST /admin/roles
  • DELETE /admin/files/{file_id}

Practical curl examples (illustrative)

• Start a chunked upload curl -X POST -F "chunk_data=@path/to/chunk1"
  -F "chunk_index=0"
  -F "total_chunks=5"
  http://localhost:5000/upload_chunk
• Complete upload curl -X POST -H "Content-Type: application/json"
  -d '{"file_id":"abc123","name":"report.txt","mime_type":"text/plain","total_size":12345}'
  http://localhost:5000/upload_complete
• Download a file curl -L http://localhost:5000/files/abc123 --output report.txt
• List files curl http://localhost:5000/files
• Rename a file curl -X POST -H "Content-Type: application/json"
  -d '{"new_name":"final_report.txt"}'
  http://localhost:5000/files/abc123/rename
• Delete a file curl -X DELETE http://localhost:5000/files/abc123

Security and access control

• Role-based access control (RBAC) is built into the app
  • Admins can manage users and files with elevated privileges
  • Regular users can upload, download, and manage their own files
• Authentication and authorization
  • The app supports session-based or token-based authentication
  • Secrets should be stored securely (e.g., environment variables, secret managers)
• Data protection
  • Use TLS in production to protect data in transit
  • Consider Redis persistence and backups for durability
  • Use access controls on Redis to limit who can read/write data
• Input validation
  • Validate file names and mime types
  • Sanitize inputs to prevent injection or path traversal
• Logging and observability
  • Implement structured logging for requests, errors, and security events
  • Monitor Redis metrics and Flask performance
• Secret rotation
  • Rotate APP_SECRET_KEY periodically
  • Use ephemeral tokens for short-lived sessions

Performance and scaling

• Chunked uploads reduce peak memory usage on the server
• Redis provides fast key-value access and helps with quick metadata operations
• Horizontal scaling
  • Run multiple Flask instances behind a load balancer
  • Use Redis as a centralized backing store
• Caching strategy
  • Cache frequently accessed metadata, but store file chunks in Redis with careful eviction policies if needed
• Persistence tuning
  • Enable Redis AOF or RDB persistence as appropriate for durability needs
  • Ensure backups of Redis data for recovery

Testing and quality

• Unit tests
  • Cover core operations like upload_chunk, upload_complete, download, rename, and delete
• Integration tests
  • Spin up a Redis instance and test end-to-end flows
• Linting and formatting
  • Use flake8 or similar linters
  • Enforce consistent formatting (black, isort)
• CI setup
  • Run tests on push and PRs
  • Build Docker images for verification

Environment configuration

• Required environment variables
  • REDIS_URL: connection string to Redis
  • APP_SECRET_KEY: secret for session or token generation
  • ADMIN_USERS: comma-separated list of admin emails or usernames
• Optional variables
  • FLASK_ENV: development or production
  • LOG_LEVEL: debug, info, warning, error
  • REDIS_PASSWORD: if Redis is password-protected
• Secrets handling
  • Do not commit secrets to version control
  • Use a secrets manager or environment-specific config files

Customization and extension

• Extending storage backends
  • Swap Redis for another storage plugin if needed
  • Implement a backend interface to support alternative stores
• Front-end integration
  • Build a small web UI to manage uploads, downloads, and file metadata
  • Integrate with a mobile or desktop client for large file transfers
• Plugins and hooks
  • Add pre- and post-operation hooks (e.g., for validation, logging, or notifications)
• Internationalization
  • Add i18n support for users in different locales

Operation and maintenance

• Backups and disaster recovery
  • Regular Redis backups
  • Asset backups for any non-Redis persisted data
• Monitoring
  • Track request latency and error rates
  • Monitor Redis health (memory usage, connections, persistence status)
• Upgrades
  • Test new versions in staging before production
  • Backward compatibility for API changes or data model changes
• Documentation
  • Keep API docs up to date
  • Document any breaking changes in a changelog

Troubleshooting

• Common issues
  • Redis connection errors: verify REDIS_URL, ensure Redis is running
  • File chunking failures: confirm total_chunks and chunk indices align
  • Permission errors: check user roles and ACLs
• Debugging tips
  • Enable verbose logs in development
  • Use curl with verbose flag (-v) to inspect requests
• Environment problems
  • Verify Python version compatibility
  • Check that environment variables are loaded correctly

Releases and updates

• The Releases page hosts packaged artifacts and release notes
• For ready-to-run assets, visit the Releases page to download a suitable package
• Link to the Releases page for quick access: https://github.com/LUC4S007/flask-redis-file-manager/releases
• You can also use the badge above to jump directly to the releases

Credits and license

• Project authors and contributors
• Acknowledgments for open-source components
• License: MIT (or your chosen license)
• How to contribute
  • Follow the contributing guidelines
  • Submit issues and pull requests with clear descriptions
  • Write tests for new features and bug fixes

Roadmap

• Planned features and enhancements
  • More robust RBAC with token scopes
  • Server-side encryption at rest for file chunks
  • Advanced search and tagging
  • Web UI for file management
  • Performance improvements for very large files
  • Observability: metrics, traces, and dashboards

Badges and topics

• Admin
• airforshare
• backend
• base64
• chunked-upload
• docker
• file-manager
• file-storage
• file-upload
• flask
• python
• redis
• redis-storage
• user-management
• web-application
• webapp

Images and visuals

• Architecture overview
  • Architecture diagram: https://picsum.photos/seed/architecture/1200/400
• Redis integration
  • Redis backbone image: https://picsum.photos/seed/redis/1200/400
• Docker deployment
  • Docker layout: https://picsum.photos/seed/docker/1200/300
• UI concept (optional)
  • Mock UI sketch: https://picsum.photos/seed/ui/1000/500

Notes on licensing and usage

• The repository is intended for learning, personal projects, and small deployments.
• If you plan to deploy in production, assess security, data protection, and compliance needs in your environment.
• Use the Releases page to obtain official assets and follow the included guidance in each release.

Changelog (quick glance)

• v0.x.y: Initial release with core features
• v0.x.y+1: Added chunked upload support and basic RBAC
• v0.x.y+2: Docker Compose setup and improved Redis integration
• v0.x.y+3: API contract improvements and better error handling

Appendix: Quick reference

• Base URL for API (example)
  • http://localhost:5000
• Redis endpoint
  • redis://localhost:6379/0
• Admin tools
  • Admin endpoints are protected and require admin credentials
• Local development tips
  • Use Docker to mimic production
  • Run tests frequently during feature development

If you need to explore the latest release assets or download a ready-to-run package, you can visit the Releases page at the link above. For quick access, the same URL is referenced again in this section, and the link is visible at the top of the document. This approach keeps you aligned with the current distribution format and ensures you always have a trusted path to the official assets.