Skip to content

Commit ecea351

Browse files
monilpatmonilpat
authoredJan 11, 2025
Merge pull request elizaOS#2110 from tomguluson92/main
feat: TTS(Text2Speech) with over 15 languages support!
2 parents 9f83833 + a78d3e3 commit ecea351

File tree

8 files changed

+749
-0
lines changed

8 files changed

+749
-0
lines changed
 

‎packages/plugin-tts/.npmignore

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
*
2+
3+
!dist/**
4+
!package.json
5+
!readme.md
6+
!tsup.config.ts
7+
!tsconfig.json

‎packages/plugin-tts/README.md

+173
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
# @elizaos/plugin-tts
2+
3+
A plugin for text-to-speech(TTS) generation using the FAL.ai API within the ElizaOS ecosystem.
4+
5+
## Description
6+
7+
The text-to-speech(TTS) plugin enables AI-powered creation of speech through FAL.ai's services. It provides functionality to generate audio from text descriptions, automatically detects language, and selects appropriate voice models.
8+
9+
## Installation
10+
11+
```bash
12+
pnpm install @elizaos/plugin-tts
13+
```
14+
15+
## Configuration
16+
17+
The plugin requires the following environment variable or runtime setting to be set:
18+
19+
```typescript
20+
FAL_API_KEY=<Your FAL.ai API key>
21+
```
22+
23+
## Usage
24+
25+
### Basic Integration
26+
27+
```typescript
28+
import { TTSGenerationPlugin } from "@elizaos/plugin-tts";
29+
```
30+
31+
### Voice Generation Examples
32+
33+
```typescript
34+
// The plugin responds to natural language commands like:
35+
36+
"Generate TTS of Hello World";
37+
"Create a TTS for Welcome to ElizaOS";
38+
"Make a TTS saying [your text]";
39+
```
40+
41+
## API Reference
42+
43+
### Actions
44+
45+
#### GENERATE_TTS
46+
47+
Generates speech audio based on text input.
48+
49+
**Aliases:**
50+
- TTS_GENERATION
51+
- CREATE_TTS
52+
- TEXT2SPEECH
53+
- T2S
54+
- TEXT_TO_SPEECH
55+
- AUDIO_CREATE
56+
57+
**Features:**
58+
- Automatic language detection
59+
- Voice selection based on detected language
60+
- Local file caching
61+
- Progress tracking
62+
- Error handling
63+
64+
## Common Issues & Troubleshooting
65+
66+
1. **Generation Failures**
67+
- Verify FAL API key is correctly set
68+
- Ensure text input is at least 3 characters long
69+
- Check network connectivity to FAL.ai services
70+
71+
2. **Storage Issues**
72+
- Verify write permissions to content_cache directory
73+
- Ensure sufficient disk space
74+
- Check if content_cache directory exists
75+
76+
## Security Best Practices
77+
78+
1. **API Key Management**
79+
- Store FAL API key securely using runtime settings or environment variables
80+
- Never commit API keys to version control
81+
- Monitor API usage
82+
83+
## Development Guide
84+
85+
### Setting Up Development Environment
86+
87+
1. Clone the repository
88+
2. Install dependencies:
89+
90+
```bash
91+
pnpm install
92+
```
93+
94+
3. Build the plugin:
95+
96+
```bash
97+
pnpm run build
98+
```
99+
100+
4. Run the plugin:
101+
102+
```bash
103+
pnpm run dev
104+
```
105+
106+
## Future Enhancements
107+
108+
1. **Advanced Voice Features**
109+
- Custom voice model support
110+
- Voice style transfer
111+
- Emotion control
112+
- Speech rate adjustment
113+
- Pitch modification
114+
- Multiple speaker support
115+
116+
2. **Audio Processing**
117+
- Background noise reduction
118+
- Audio quality enhancement
119+
- Format conversion options
120+
- Volume normalization
121+
- Audio effects processing
122+
- Batch processing support
123+
124+
3. **Language Support**
125+
- Expanded language detection
126+
- Regional accent support
127+
- Dialect customization
128+
- Pronunciation improvements
129+
- Multi-language mixing
130+
- Custom pronunciation rules
131+
132+
4. **Integration Features**
133+
- Streaming audio support
134+
- Real-time generation
135+
- Caching optimization
136+
- Batch generation
137+
- Queue management
138+
- Progress monitoring
139+
140+
5. **Developer Tools**
141+
- Extended API options
142+
- Testing framework
143+
- Performance profiling
144+
- Debug logging
145+
- Integration examples
146+
- Documentation generator
147+
148+
We welcome community feedback and contributions to help prioritize these enhancements.
149+
150+
## Contributing
151+
152+
Contributions are welcome! Please see the [CONTRIBUTING.md](CONTRIBUTING.md) file for more information.
153+
154+
## Credits
155+
156+
This plugin integrates with and builds upon several key technologies:
157+
158+
- [FAL.ai](https://fal.ai/): AI model deployment platform
159+
- [langdetect](https://github.com/wooorm/franc): Language detection library
160+
- [ElizaOS](https://elizaos.com): Core framework
161+
162+
Special thanks to:
163+
- The FAL.ai team for AI infrastructure
164+
- The langdetect development community
165+
- The Eliza community for their contributions and feedback
166+
167+
For more information about TTS capabilities:
168+
- [FAL.ai Documentation](https://fal.ai/docs)
169+
- [ElizaOS Documentation](https://docs.elizaos.com)
170+
171+
## License
172+
173+
This plugin is part of the Eliza project. See the main project repository for license information.

‎packages/plugin-tts/eslint.config.mjs

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
import eslintGlobalConfig from "../../eslint.config.mjs";
2+
3+
export default [...eslintGlobalConfig];

‎packages/plugin-tts/package.json

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
{
2+
"name": "@elizaos/plugin-tts",
3+
"version": "0.1.7",
4+
"type": "module",
5+
"main": "dist/index.js",
6+
"module": "dist/index.js",
7+
"types": "dist/index.d.ts",
8+
"exports": {
9+
"./package.json": "./package.json",
10+
".": {
11+
"import": {
12+
"@elizaos/source": "./src/index.ts",
13+
"types": "./dist/index.d.ts",
14+
"default": "./dist/index.js"
15+
}
16+
}
17+
},
18+
"files": [
19+
"dist"
20+
],
21+
"dependencies": {
22+
"@elizaos/core": "workspace:*",
23+
"tsup": "8.3.5",
24+
"whatwg-url": "7.1.0"
25+
},
26+
"scripts": {
27+
"build": "tsup --format esm --dts",
28+
"dev": "tsup --format esm --dts --watch",
29+
"lint": "eslint --fix --cache ."
30+
},
31+
"peerDependencies": {
32+
"whatwg-url": "7.1.0"
33+
}
34+
}

0 commit comments

Comments
 (0)