Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Sign up
Appearance settings

samuelson-chen/airi

Folders and files

NameName
Last commit message
Last commit date

Latest commit

History

2,887 Commits

Repository files navigation

Project AIRI

Re-creating Neuro-sama, a soul container of AI waifu / virtual characters to bring them into our world.

[Join Discord Server] [Try it] [简体中文] [日本語] [Русский] [Tiếng Việt] [Français] [한국어]

AIRI - A container of cyber living souls, re-creation of Neuro-sama | Product Hunt moeru-ai%2Fairi | Trendshift

Heavily inspired by Neuro-sama

Warning

Attention: We do not have any officially minted cryptocurrency or token associated with this project. Please check the information and proceed with caution.

Note

We've got a whole dedicated organization @proj-airi for all the sub-projects born from Project AIRI. Check it out!

RAG, memory system, embedded database, icons, Live2D utilities, and more!

Tip

We have a translation project on Crowdin. If you find any inaccurate translations, feel free to contribute improvements there. Crowdin | Agile localization for tech companies

Have you dreamed about having a cyber living being (cyber waifu, digital pet) or digital companion that could play with and talk to you?

With the power of modern large language models like ChatGPT and famous Claude, asking a virtual being to roleplay and chat with us is already easy enough for everyone. Platforms like Character.ai (a.k.a. c.ai) and JanitorAI as well as local playgrounds like SillyTavern are already good-enough solutions for a chat based or visual adventure game like experience.

But, what about the abilities to play games? And see what you are coding at? Chatting while playing games, watching videos, and is capable of doing many other things.

Perhaps you know Neuro-sama already. She is currently the best virtual streamer capable of playing games, chatting, and interacting with you and the participants. Some also call this kind of being "digital human." Sadly, as it's not open sourced, you cannot interact with her after her live streams go offline.

Therefore, this project, AIRI, offers another possibility here: let you own your digital life, cyber living, easily, anywhere, anytime.

DevLogs We Posted & Recent Updates

What's So Special About This Project?

Unlike the other AI driven VTuber open source projects, アイリ was built with support of many Web technologies such as WebGPU, WebAudio, Web Workers, WebAssembly, WebSocket, etc. from the first day.

Tip

Worrying about the performance drop since we are using Web related technologies?

Don't worry, while Web browser version is meant to give an insight about how much we can push and do inside browsers, and webviews, we will never fully rely on this, the desktop version of AIRI is capable of using native NVIDIA CUDA and Apple Metal by default (thanks to HuggingFace & beloved candle project), without any complex dependency managements, considering the tradeoff, it was partially powered by Web technologies for graphics, layouts, animations, and the WIP plugin systems for everyone to integrate things.

This means that アイリ is capable of running on modern browsers and devices and even on mobile devices (already done with PWA support). This brings a lot of possibilities for us (the developers) to build and extend the power of アイリ VTuber to the next level, while still leaving the flexibilities for users to enable features that requires TCP connections or other non-Web technologies such as connecting to a Discord voice channel or playing Minecraft and Factorio with friends.

Note

We are still in the early stage of development where we are seeking out talented developers to join us and help us to make アイリ a reality.

It's ok if you are not familiar with Vue.js, TypeScript, and devtools required for this project, you can join us as an artist, designer, or even help us to launch our first live stream.

Even if you are a big fan of React, Svelte or even Solid, we welcome you. You can open a sub-directory to add features that you want to see in アイリ, or would like to experiment with.

Fields (and related projects) that we are looking for:

  • Live2D modeller
  • VRM modeller
  • VRChat avatar designer
  • Computer Vision
  • Reinforcement Learning
  • Speech Recognition
  • Speech Synthesis
  • ONNX Runtime
  • Transformers.js
  • vLLM
  • WebGPU
  • Three.js
  • WebXR (checkout the another project we have under the @moeru-ai organization)

If you are interested, why not introduce yourself here? Would like to join part of us to build AIRI?

Current Progress

Capable of

  • Brain
  • Ears
    • Audio input from browser
    • Audio input from Discord
    • Client side speech recognition
    • Client side talking detection
  • Mouth
  • Body
    • VRM support
      • Control VRM model
    • VRM model animations
      • Auto blink
      • Auto look at
      • Idle eye movement
    • Live2D support
      • Control Live2D model
    • Live2D model animations
      • Auto blink
      • Auto look at
      • Idle eye movement

Development

For detailed instructions to develop this project, follow CONTRIBUTING.md

Note

By default, pnpm dev will start the development server for the Stage Web (browser version). If you would like to try developing the desktop version, please make sure you read CONTRIBUTING.md to setup the environment correctly.

pnpm i
pnpm dev

Stage Web (Browser Version at airi.moeru.ai)

pnpm dev

Stage Tamagotchi (Desktop Version)

pnpm dev:tamagotchi

A Nix package for Tamagotchi is included. To run airi with Nix, first make sure to enable flakes, then run:

nix run github:moeru-ai/airi

Stage Pocket (Mobile Version)

Start the development server for the capacitor:

pnpm dev:pocket:ios <DEVICE_ID_OR_SIMULATOR_NAME>
# Or
CAPACITOR_DEVICE_ID=<DEVICE_ID_OR_SIMULATOR_NAME> pnpm dev:pocket:ios

You can see the list of available devices and simulators by running pnpm exec cap run ios --list.

If you need to connect server channel on pocket in wireless mode, you need to start tamagotchi as root:

sudo pnpm dev:tamagotchi

Then enable secure websocket in tamagotchi settings/system/general.

Documentation Site

pnpm dev:docs

Publish

Please update the version in Cargo.toml after running bumpp:

npx bumpp --no-commit --no-tag

Support of LLM API Providers (powered by xsai)

Sub-projects Born from This Project

%%{ init: { 'flowchart': { 'curve': 'catmullRom' } } }%%
flowchart TD
 Core("Core")
 Unspeech("unspeech")
 DBDriver("@proj-airi/drizzle-duckdb-wasm")
 MemoryDriver("[WIP] Memory Alaya")
 DB1("@proj-airi/duckdb-wasm")
 SVRT("@proj-airi/server-runtime")
 Memory("Memory")
 STT("STT")
 Stage("Stage")
 StageUI("@proj-airi/stage-ui")
 UI("@proj-airi/ui")
 subgraph AIRI
 DB1 --> DBDriver --> MemoryDriver --> Memory --> Core
 UI --> StageUI --> Stage --> Core
 Core --> STT
 Core --> SVRT
 end
 subgraph UI_Components
 UI --> StageUI
 UITransitions("@proj-airi/ui-transitions") --> StageUI
 UILoadingScreens("@proj-airi/ui-loading-screens") --> StageUI
 FontCJK("@proj-airi/font-cjkfonts-allseto") --> StageUI
 FontXiaolai("@proj-airi/font-xiaolai") --> StageUI
 end
 subgraph Apps
 Stage --> StageWeb("@proj-airi/stage-web")
 Stage --> StageTamagotchi("@proj-airi/stage-tamagotchi")
 Core --> RealtimeAudio("@proj-airi/realtime-audio")
 Core --> PromptEngineering("@proj-airi/playground-prompt-engineering")
 end
 subgraph Server_Components
 Core --> ServerSDK("@proj-airi/server-sdk")
 ServerShared("@proj-airi/server-shared") --> SVRT
 ServerShared --> ServerSDK
 end
 STT -->|Speaking| Unspeech
 SVRT -->|Playing Factorio| F_AGENT
 SVRT -->|Playing Minecraft| MC_AGENT
 subgraph Factorio_Agent
 F_AGENT("Factorio Agent")
 F_API("Factorio RCON API")
 factorio-server("factorio-server")
 F_MOD1("autorio")
 F_AGENT --> F_API -.-> factorio-server
 F_MOD1 -.-> factorio-server
 end
 subgraph Minecraft_Agent
 MC_AGENT("Minecraft Agent")
 Mineflayer("Mineflayer")
 minecraft-server("minecraft-server")
 MC_AGENT --> Mineflayer -.-> minecraft-server
 end
 XSAI("xsAI") --> Core
 XSAI --> F_AGENT
 XSAI --> MC_AGENT
 Core --> TauriMCP("@proj-airi/tauri-plugin-mcp")
 Memory_PGVector("@proj-airi/memory-pgvector") --> Memory
 style Core fill:#f9d4d4,stroke:#333,stroke-width:1px
 style AIRI fill:#fcf7f7,stroke:#333,stroke-width:1px
 style UI fill:#d4f9d4,stroke:#333,stroke-width:1px
 style Stage fill:#d4f9d4,stroke:#333,stroke-width:1px
 style UI_Components fill:#d4f9d4,stroke:#333,stroke-width:1px
 style Server_Components fill:#d4e6f9,stroke:#333,stroke-width:1px
 style Apps fill:#d4d4f9,stroke:#333,stroke-width:1px
 style Factorio_Agent fill:#f9d4f2,stroke:#333,stroke-width:1px
 style Minecraft_Agent fill:#f9d4f2,stroke:#333,stroke-width:1px
 style DBDriver fill:#f9f9d4,stroke:#333,stroke-width:1px
 style MemoryDriver fill:#f9f9d4,stroke:#333,stroke-width:1px
 style DB1 fill:#f9f9d4,stroke:#333,stroke-width:1px
 style Memory fill:#f9f9d4,stroke:#333,stroke-width:1px
 style Memory_PGVector fill:#f9f9d4,stroke:#333,stroke-width:1px
Loading

Similar Projects

Open sourced ones

Non-open-sourced ones

Project Status

Repobeats analytics image

Acknowledgements

Star History

Star History Chart

About

💖🧸 Self hosted, you-owned Grok Companion, a container of souls of waifu, cyber livings to bring them into our worlds, wishing to achieve Neuro-sama's altitude. Capable of realtime voice chat, Minecraft, Factorio playing. Web / macOS / Windows supported.

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

Contributors

Languages

  • TypeScript 58.5%
  • Vue 37.1%
  • JavaScript 1.4%
  • Rust 1.3%
  • CSS 0.9%
  • HTML 0.5%
  • Other 0.3%

AltStyle によって変換されたページ (->オリジナル) /