Compare commits
1 Commits
bt-new-mod
...
more-clean
| Author | SHA1 | Date |
|---|---|---|
Brandon Thomas
|
2b39f5d02e |
|
|
@ -1,16 +0,0 @@
|
|||
{
|
||||
"files.exclude": {
|
||||
"**/*~": true,
|
||||
".sqlx/**": true,
|
||||
"_database/**": true,
|
||||
"_database/elasticsearch/**": true,
|
||||
"_database/sql/**": true,
|
||||
"frontend_old/**": true,
|
||||
"includes/**": true,
|
||||
"target/**": true,
|
||||
"test_data/**": true,
|
||||
"tmp/**": true,
|
||||
"virtualenv/**": true,
|
||||
"windows_dlls/**": true,
|
||||
}
|
||||
}
|
||||
|
|
@ -1,26 +0,0 @@
|
|||
Code Conventions
|
||||
================
|
||||
|
||||
We use three languages at Artcraft: Rust, TypeScript, and Python.
|
||||
|
||||
## Rust style guide
|
||||
|
||||
We deviate from the official Rust style guide in a few places:
|
||||
|
||||
- We use two spaces for indentation instead of four.
|
||||
- We typically keep one function or type definition per file, though this isn't a steadfast rule.
|
||||
- We use a Cargo workspace to share code within our monorepo.
|
||||
|
||||
### Rust Preferred Libraries
|
||||
|
||||
We use the following Rust libraries, though we are open to migrating to alternatives:
|
||||
|
||||
- `actix-web` for our HTTP web server
|
||||
- `reqwest` for HTTP client requests
|
||||
- `sqlx` for SQL database operations
|
||||
- `tokio` for asynchronous programming
|
||||
|
||||
## TypeScript style guide
|
||||
|
||||
- We use two spaces for indentation instead of four.
|
||||
- We use `nx` for monorepo management
|
||||
|
|
@ -1,105 +0,0 @@
|
|||
# ArtCraft Server
|
||||
|
||||
ArtCraft's server is a Rust / Actix app called `storyteller-web`.
|
||||
|
||||
You don't need to run this to develop the ArtCraft application, but it can be useful to spin up
|
||||
a development instance for adding new server functions or as your own private local copy.
|
||||
|
||||
`storyteller-web` can function entirely within the development environment, and ArtCraft can be
|
||||
pointed to a local instance of the server.
|
||||
|
||||
**TODO**: Concisely describe setting up server components. Note: we have
|
||||
previous docs in `old/` that may still be relevant, though they're perhaps
|
||||
slightly out of date.
|
||||
|
||||
## Server Setup (Mac)
|
||||
|
||||
### Install and Migrate MySQL Database
|
||||
|
||||
```bash
|
||||
# For now, the tooling doesn't support MySQL 9.6 as it was just released in January 2026.
|
||||
# This will probably change in the future to no longer need to specify the 8.* series.
|
||||
brew install mysql@8.4
|
||||
|
||||
# Try to connect
|
||||
mysql -uroot
|
||||
|
||||
# If connection fails, start service, then try again.
|
||||
brew services start mysql
|
||||
mysql -uroot
|
||||
|
||||
# If that fails, reboot and try connecting again.
|
||||
```
|
||||
|
||||
Create the `storyteller` database table. Connect to the database:
|
||||
|
||||
```bash
|
||||
sudo mysql -u root
|
||||
|
||||
# If it needs a password, use argument -p and when prompted use "root" as the typical default password
|
||||
```
|
||||
|
||||
Then run this:
|
||||
|
||||
```mysql
|
||||
use mysql;
|
||||
CREATE DATABASE storyteller;
|
||||
CREATE USER 'storyteller'@'localhost' IDENTIFIED BY 'password';
|
||||
GRANT ALL PRIVILEGES ON storyteller.* TO 'storyteller'@'localhost';
|
||||
```
|
||||
|
||||
Then verify access with `./script/dev_mysql_connect.sh`
|
||||
|
||||
```bash
|
||||
cargo install diesel_cli \
|
||||
--no-default-features \
|
||||
--features mysql,sqlite
|
||||
```
|
||||
|
||||
If this fails,
|
||||
|
||||
```bash
|
||||
brew install zstd
|
||||
brew install openssl
|
||||
```
|
||||
|
||||
Possibly,
|
||||
|
||||
```bash
|
||||
export RUSTFLAGS="-L$(brew --prefix zstd)/lib -L$(brew --prefix openssl)/lib"
|
||||
```
|
||||
|
||||
In older Macs, there [have been some issues with Diesel CLI](https://github.com/diesel-rs/diesel/issues/2605)
|
||||
that require a few extra dependencies to be installed:
|
||||
|
||||
```bash
|
||||
# If you're on Mac and the above command didn't work, run the following and then retry:
|
||||
brew install libpq
|
||||
```
|
||||
|
||||
You're ready to run the migrations:
|
||||
|
||||
```bash
|
||||
diesel migration run
|
||||
```
|
||||
|
||||
You might get a scary message about `"Encountered unknown type for Mysql: enum"` -- you can safely ignore this
|
||||
error if you see it in isolation. It doesn't impact the migrations whatsoever.
|
||||
|
||||
Finally, you'll need the sqlx CLI tool to run codegen.
|
||||
You likely won't need this now, but if you change any queries, this will be necessary:
|
||||
|
||||
```bash
|
||||
cargo install sqlx-cli --features rustls,mysql,sqlite
|
||||
```
|
||||
|
||||
### Install Redis (Mac)
|
||||
|
||||
```bash
|
||||
# Install Redis
|
||||
brew install redis
|
||||
|
||||
# Start the service:
|
||||
brew services start redis
|
||||
```
|
||||
|
||||
|
|
@ -1,395 +0,0 @@
|
|||
Local Development Setup Guide (for bare metal)
|
||||
==============================================
|
||||
|
||||
This document describes how to run the FakeYou infrastructure on your development
|
||||
machine's bare metal (no containers).
|
||||
|
||||
Setup instructions and development were tested and known to work on the following
|
||||
platforms:
|
||||
|
||||
- Mac OS (Apple M2 silicon)
|
||||
- Ubuntu 22.04
|
||||
- Windows Subsystem for Linux (Ubuntu 22.04 distro) aka "WSL"
|
||||
|
||||
The applications (binary targets) you might be interested in running include, but
|
||||
are not limited to:
|
||||
|
||||
- `storyteller-web`, the HTTP API server
|
||||
- `download-job`, which downloads models from the web and stores them in GCS
|
||||
- `inference-job`, which downloads models from GCS and runs inference with them
|
||||
|
||||
Setup Instructions
|
||||
------------------
|
||||
|
||||
### (1) Install Rust
|
||||
|
||||
Install Rust [using this guide](https://www.rust-lang.org/learn/get-started). If it asks,
|
||||
you'll want "stable" Rust, not "nightly" Rust. If it doesn't ask, it defaults to "stable".
|
||||
|
||||
For Windows Subsystem for Linux, you'll want the WSL version, not the `.exe` file!
|
||||
|
||||
### (2) Install the necessary libraries
|
||||
|
||||
#### Mac
|
||||
|
||||
On an up-to-date M1/M2 Mac, there should be no libraries you need to install! Any exceptions
|
||||
to this assumption should be documented here.
|
||||
|
||||
#### Ubuntu 22.04 and WSL:
|
||||
|
||||
If you're on Ubuntu or WSL, you'll need to install some libraries to build the Rust code:
|
||||
|
||||
```bash
|
||||
# If you haven't run apt before, you'll need to fetch the package list:
|
||||
sudo apt update
|
||||
|
||||
# Install the dependencies
|
||||
sudo apt install jq \
|
||||
libmysqlclient-dev \
|
||||
libsqlite3-dev \
|
||||
pkgconf
|
||||
```
|
||||
|
||||
(Note: jq is for combining mysql codegen outputs; pkgconf is if using openssl instead of rustls.)
|
||||
|
||||
### (3) Install MySQL server
|
||||
|
||||
#### Mac
|
||||
|
||||
```bash
|
||||
# Install MySQL
|
||||
brew install mysql
|
||||
|
||||
# Try to connect
|
||||
mysql -uroot
|
||||
```
|
||||
|
||||
In the event of failure to connect, try the following steps:
|
||||
|
||||
```bash
|
||||
# If connection fails, start service
|
||||
brew services start mysql
|
||||
|
||||
# Try to connect again
|
||||
# If this fails, reboot the machine, eg. using `sudo reboot now`
|
||||
mysql -uroot
|
||||
```
|
||||
|
||||
#### Ubuntu 22.04 and WSL
|
||||
|
||||
```bash
|
||||
# Install MySQL
|
||||
sudo apt install mysql-server
|
||||
|
||||
# Try to connect
|
||||
# The default password is typically "root".
|
||||
sudo mysql -u root -p
|
||||
````
|
||||
|
||||
In the event of failure to connect, try the following steps:
|
||||
|
||||
```bash
|
||||
# If it failed, you may need to start MySQL (typically only on WSL, not on native Ubuntu)
|
||||
sudo service mysql start
|
||||
|
||||
# You may also have to change some socket file permissions (again, typically only on WSL)
|
||||
sudo chmod g+rx /var/run/mysqld
|
||||
sudo usermod -aG mysql $USER
|
||||
newgrp mysql
|
||||
|
||||
# Try to connect again
|
||||
# The default password is typically "root".
|
||||
# If this fails, try rebooting. If it still fails, read some of the suggestions at the bottom
|
||||
# of this README.
|
||||
sudo mysql -u root -p
|
||||
```
|
||||
|
||||
### (4) Install a `storyteller` user and table in MySQL
|
||||
|
||||
First, connect to MySQL:
|
||||
|
||||
```bash
|
||||
# If the following command asks for a password, the password is typically "root"
|
||||
sudo mysql -u root -p
|
||||
```
|
||||
|
||||
Once in MySQL, run the following to set up the database and user accounts:
|
||||
|
||||
```mysql
|
||||
use mysql;
|
||||
CREATE DATABASE storyteller;
|
||||
CREATE USER 'storyteller'@'localhost' IDENTIFIED BY 'password';
|
||||
GRANT ALL PRIVILEGES ON storyteller.* TO 'storyteller'@'localhost';
|
||||
```
|
||||
|
||||
Then verify access with `./dev_mysql_connect.sh`
|
||||
|
||||
### (5) Install Diesel CLI (for MySQL migrations):
|
||||
|
||||
In order to run MySQL database migrations, you'll need to install the migration tool we use:
|
||||
|
||||
```bash
|
||||
cargo install diesel_cli \
|
||||
--no-default-features \
|
||||
--features mysql,sqlite
|
||||
```
|
||||
|
||||
In older Macs, there [have been some issues with Diesel CLI](https://github.com/diesel-rs/diesel/issues/2605)
|
||||
that require a few extra dependencies to be installed:
|
||||
|
||||
```bash
|
||||
# If you're on Mac and the above command didn't work, run the following and then retry:
|
||||
brew install libpq
|
||||
```
|
||||
|
||||
### (6) Run the pending database migrations:
|
||||
|
||||
You're ready to run the migrations:
|
||||
|
||||
```bash
|
||||
diesel migration run
|
||||
```
|
||||
|
||||
You might get a scary message about `"Encountered unknown type for Mysql: enum"` -- you can safely ignore this
|
||||
error if you see it in isolation. It doesn't impact the migrations whatsoever. (See notes at the bottom of
|
||||
the README for details.)
|
||||
|
||||
### (7) (optional) Install the SQLx CLI (if doing database development)
|
||||
|
||||
(You can probably skip this step unless you're a backend engineer.)
|
||||
|
||||
If (and only if) you're doing Rust service development that changes the MySQL queries, then you should install
|
||||
SQLx utilities. This tool is used to cache the column types and query plans for CI, as SQLx statically verifies
|
||||
raw MySQL queries against the schema and Rust types.
|
||||
|
||||
We'll be using diesel to manage the migrations, but sqlx within the app to actually perform queries.
|
||||
Diesel is an immature ORM, which isn't a good tech bet, so we use sqlx as at-compile-time
|
||||
typesafe SQL.
|
||||
|
||||
(TODO: Reevaluate this assessment for 2023-2024 and beyond. Maybe we do want Diesel now.)
|
||||
|
||||
```bash
|
||||
# NB(bt,2023-10-04): "--no-default-features" not working
|
||||
# cargo install sqlx-cli --no-default-features --features rustls,mysql,sqlite
|
||||
cargo install sqlx-cli --features rustls,mysql,sqlite
|
||||
```
|
||||
|
||||
### (8) Install hosts file:
|
||||
|
||||
(This is the same for Linux, Mac, and WSL.)
|
||||
|
||||
If you're developing against the frontend, it'll target development domains (eg. `dev.fakeyou.com`) instead
|
||||
of `127.0.0.1` or `localhost`. You can make your machine route domains to localhost by editing your hosts
|
||||
file (located at `/etc/hosts`) to include the following configuration lines:
|
||||
|
||||
```
|
||||
# 1) Edit hosts file with your editor of choice
|
||||
#
|
||||
# - sudo vi /etc/hosts
|
||||
# - sudo nano /etc/hosts
|
||||
#
|
||||
# 2) Then paste:
|
||||
#
|
||||
|
||||
127.0.0.1 dev.fakeyou.com
|
||||
127.0.0.1 api.dev.fakeyou.com
|
||||
127.0.0.1 devproxy.fakeyou.com
|
||||
|
||||
127.0.0.1 dev.storyteller.ai
|
||||
127.0.0.1 api.dev.storyteller.ai
|
||||
127.0.0.1 devproxy.storyteller.ai
|
||||
```
|
||||
|
||||
### (9) Install Redis
|
||||
|
||||
#### Mac:
|
||||
|
||||
```bash
|
||||
# Install Redis
|
||||
brew install redis
|
||||
|
||||
# Start the service:
|
||||
brew services start redis
|
||||
```
|
||||
|
||||
#### Ubuntu 22.04 and WSL
|
||||
|
||||
```
|
||||
# Install Redis
|
||||
sudo apt install redis
|
||||
|
||||
# Start the service (typically only needed on WSL):
|
||||
sudo service redis-server start
|
||||
```
|
||||
|
||||
### (10) Install Elasticsearch (optional)
|
||||
|
||||
(You can most likely skip this step.)
|
||||
|
||||
Mac: (TODO - haven't installed yet)
|
||||
|
||||
WSL: (TODO - haven't installed yet)
|
||||
|
||||
Ubuntu 22.04:
|
||||
|
||||
Install:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elastic.gpg
|
||||
|
||||
echo "deb [signed-by=/usr/share/keyrings/elastic.gpg] https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
|
||||
|
||||
sudo apt update
|
||||
sudo apt install elasticsearch
|
||||
```
|
||||
|
||||
Configure:
|
||||
|
||||
```text
|
||||
# Edit /etc/elasticsearch/elasticsearch.yml
|
||||
# Set the following:
|
||||
network.host: localhost
|
||||
```
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
sudo systemctl start elasticsearch
|
||||
sudo systemctl enable elasticsearch
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
curl -X GET 'http://localhost:9200'
|
||||
```
|
||||
|
||||
Install Kibana (prototyping UI):
|
||||
|
||||
```bash
|
||||
# Install
|
||||
sudo apt-get install kibana
|
||||
|
||||
# Run
|
||||
sudo /bin/systemctl daemon-reload
|
||||
sudo /bin/systemctl enable kibana.service
|
||||
sudo systemctl start kibana.service
|
||||
|
||||
# Access the Kibana Status Page:
|
||||
# http://localhost:5601/status
|
||||
|
||||
# Access the query console UI:
|
||||
# Click Sidebar > Management > Dev Tools > Console
|
||||
# http://localhost:5601/
|
||||
```
|
||||
|
||||
Extra reading on setting up Elasticsearch:
|
||||
|
||||
- [Guide 1](https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-elasticsearch-on-ubuntu-22-04)
|
||||
- [Guide 2](https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html)
|
||||
- [Kibana](https://www.elastic.co/guide/en/kibana/current/deb.html)
|
||||
|
||||
### (11) Run one or more applications:
|
||||
|
||||
To start the HTTP API server,
|
||||
|
||||
```bash
|
||||
cargo run --bin storyteller-web
|
||||
```
|
||||
|
||||
Note that this compiles and runs the "development" binary. It's faster and easier to debug than the
|
||||
optimized "production" build. To build a fully optimized production release,
|
||||
run `cargo build --release --bin storyteller-web` . Note that this will take much longer.
|
||||
|
||||
You may notice that this might not work right off the bat. You may need to specify some environment variables
|
||||
and supply some development secrets.
|
||||
|
||||
Secrets:
|
||||
|
||||
- Global secrets (ask Brandon for these):
|
||||
- Put `.env-secrets` in the root `storyteller-rust` directory.
|
||||
|
||||
- Per app secrets (ask Brandon for these):
|
||||
- Put `storteller-web.development-secrets.env` in `crates/service/web/storyteller_web/config`
|
||||
- Put `inference-job.development-secrets.env` in `crates/service/job/inference_job/config`
|
||||
- Put `donwload-job.development-secrets.env` in `crates/service/job/download_job/config`
|
||||
|
||||
**DO NOT CHECK THESE SECRET FILES INTO THE REPO!**
|
||||
|
||||
Environment config:
|
||||
|
||||
You may need to set local development paths for your environment. You can set these in `.env-secrets` too,
|
||||
so that they won't be updated for other engineers.
|
||||
|
||||
```
|
||||
# this should be the directory containing all of the "storyteller" projects:
|
||||
STORYTELLER_ROOT = "/home/tensor/code/storyteller" # change this
|
||||
|
||||
# The following are entirely optional, but if you want to fine tune where each project lives, you may do so:
|
||||
STORYTELLER_FRONTEND = "/somewhere/else/storyteller-frontend"
|
||||
STORYTELLER_ML = "storyteller-ml"
|
||||
STORYTELLER_RUST = "/Users/bt/Development/storyteller-rust"
|
||||
```
|
||||
|
||||
To download some ML models, run:
|
||||
|
||||
```bash
|
||||
cargo run --bin download-job
|
||||
cargo run --bin tts-download-job
|
||||
```
|
||||
|
||||
If you want to run both the HTTP API and the jobs, you'll need to run both processes in different
|
||||
terminals or tmux sessions.
|
||||
|
||||
Note also that the configurations pointing to the ML monorepo must be set up for each application.
|
||||
Additionally, some development secrets may be needed (ask the team to share).
|
||||
|
||||
To execute ML inference, run:
|
||||
|
||||
```bash
|
||||
cargo run --bin inference-job
|
||||
cargo run --bin tts-inference-job
|
||||
```
|
||||
|
||||
Again, note that the configurations pointing to the ML monorepo must be set up for each application.
|
||||
Additionally, some development secrets may be needed (ask the team to share).
|
||||
|
||||
Seeding the Database with Some Values
|
||||
-------------------------------------
|
||||
For voice designer you might want some pre-seeded audio that will hit the google cloud bucket, it will require env-secrets ask Brandon.
|
||||
cargo run --bin dev-database-seed -- --bucket
|
||||
|
||||
Solutions to Common Setup Problems
|
||||
-----------------------------------
|
||||
|
||||
### [Fix] Sqlx Error when Performing Schema Migrations
|
||||
|
||||
You might get this error message during migration,
|
||||
|
||||
```
|
||||
Encountered unknown type for Mysql: enum
|
||||
thread 'main' panicked at 'internal error: entered unreachable code: Mysql only supports a closed set of types.
|
||||
If you ever see this error message please open an issue at https://github.com/diesel-rs/diesel
|
||||
containing a dump of your schema definition.', diesel_cli/src/print_schema.rs:310:17
|
||||
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
|
||||
```
|
||||
|
||||
I haven't found the cause (it doesn't happen on newer installs), but the migrations appear to work regardless
|
||||
of this error message. You can essentially ignore it.
|
||||
|
||||
### [Fix] Can't connect to local MySQL after install on Ubuntu
|
||||
|
||||
- If MySql in local dev can't be connected to, [reset the accounts](https://linuxconfig.org/how-to-reset-root-mysql-mariadb-password-on-ubuntu-20-04-focal-fossa-linux).
|
||||
|
||||
### API Server
|
||||
```
|
||||
cargo run --bin docs
|
||||
```
|
||||
This setups an api server with swagger docs to show you responses and requests with each associated endpoint
|
||||
It is run at http://127.0.0.1:8989/
|
||||
To obtain the json for the postman collection go to this url http://127.0.0.1:8989/api-docs/openapi.json
|
||||
We use this: `https://github.com/juhaku/utoipa`
|
||||
The documentation example is in:
|
||||
`crates/service/web/storyteller_web/src/http_server/endpoints/voice_designer/inference/enqueue_tts_request.rs`
|
||||
Write into the api doc to:
|
||||
`crates/service/web/storyteller_web/src/api_doc.rs`
|
||||
|
|
@ -1,54 +0,0 @@
|
|||
README
|
||||
======
|
||||
|
||||
This directory contains a collection of (hopefully organized and up to date) documentation on
|
||||
running the various backend pieces, development setup, production maintenance, etc. Check out
|
||||
all the markdown documents in this directory tree for information organized by topic.
|
||||
|
||||
Development / Setup
|
||||
-------------------
|
||||
|
||||
- [Most of the general documents for cross-cutting setup of FakeYou.com live here.](./local_development)
|
||||
- Some per-application documents (such as for `aichatbot-sidecar`) live as READMEs under that particular
|
||||
application's code directory, eg. `/crates/{appCategory}/{appName}/README.md` (These are irrelevant
|
||||
for most folks.)
|
||||
|
||||
Web API Overview
|
||||
----------------
|
||||
|
||||
### FakeYou.com
|
||||
|
||||
* [FakeYou.com](https://fakeyou.com) is powered by the following applications:
|
||||
* HTTP APIs:
|
||||
* `storyteller-web` - HTTP API + database monolith for FakeYou
|
||||
* Asynchronous Jobs:
|
||||
* `download-job` - Handle async download of ML models (RVC, SVC, etc.)
|
||||
* `inference-job` - Run async ML model inference (RVC, SVC, SadTalker, etc.)
|
||||
This will handle most model inference going forward.
|
||||
* `tts-download-job` - (*deprecated*) Async download of Tacotron2 models only
|
||||
* `tts-inference-job` - (*deprecated*) Async TTS inference of Tacotron2 models only
|
||||
* `w2l-download-job` - (*deprecated*) Async download of videos and images
|
||||
* `w2l-inference-job` - (*deprecated*) Async W2L inference
|
||||
* Documentation Server:
|
||||
* `docs` - this will run a local swagger server with the endpoints that you have access to, http://127.0.0.1:8989/api-docs/openapi.json use this to port everything over to the post man.
|
||||
|
||||
### PowerStream
|
||||
|
||||
* [PowerStream](https://power.stream) (deprecated) is powered by the following applications:
|
||||
* HTTP APIs:
|
||||
* `storyteller-web` - HTTP + database monolith for FakeYou ***and*** PowerStream.
|
||||
* Twitch API integration:
|
||||
* `twitch-pubsub-subscriber` - Dynamically subscribes to Twitch PubSub for "active" users
|
||||
* Websocket events:
|
||||
* `websocket-gateway` - Websocket for users' browsers to pull down enriched Twitch events.
|
||||
|
||||
TODO
|
||||
----
|
||||
|
||||
Notes / TODOs:
|
||||
|
||||
* Examples for good Actix+Sqlx Tests:
|
||||
https://stackoverflow.com/questions/65370752/how-do-i-create-an-actix-web-server-that-accepts-both-sqlx-database-pools-and-tr
|
||||
|
||||
* Actix/sqlx runtime compat:
|
||||
https://github.com/launchbadge/sqlx/issues/1117#issuecomment-801237734
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
development notes
|
||||
=================
|
||||
|
||||
Actix notes
|
||||
-----------
|
||||
|
||||
json request
|
||||
```
|
||||
async fn handler(request: web::Json<Mytype>) -> impl Responder { "whatever" }
|
||||
```
|
||||
|
||||
form-multipart request
|
||||
```
|
||||
async fn handler(request: web::Form<Mytype>) -> impl Responder { "whatever" }
|
||||
```
|
||||
|
||||
route parameters
|
||||
```
|
||||
#[get("/{name}")]
|
||||
async fn hello(name: web::Path<String>) -> impl Responder { format!("Hi {}", name) }
|
||||
```
|
||||
|
|
@ -1,43 +0,0 @@
|
|||
Migrations guide
|
||||
================
|
||||
|
||||
## Diesel Migrations
|
||||
|
||||
To run pending migrations, execute the following:
|
||||
|
||||
```bash
|
||||
diesel migration run
|
||||
```
|
||||
|
||||
To revert the last 3 migrations:
|
||||
|
||||
```bash
|
||||
diesel migration revert -n 3
|
||||
```
|
||||
|
||||
To reset the entire database (drop, migrate), run:
|
||||
|
||||
```bash
|
||||
diesel database reset
|
||||
```
|
||||
|
||||
## Server Query Codegen
|
||||
|
||||
We use SQLx instead of Diesel in the production server. It's typesafe
|
||||
SQL instead of an ORM like Diesel.
|
||||
|
||||
SQLx connects to a database to derive type information, but obviously
|
||||
cannot do this for builds in CI. In order to cache the types, we build
|
||||
and check in a cache file (necessary for builds):
|
||||
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo sqlx prepare
|
||||
```
|
||||
|
||||
Now that we have multiple binaries, it's required to include all the queries in the main
|
||||
binary so we can generate the cached queries as a single target. That's then executed
|
||||
with:
|
||||
|
||||
```bash
|
||||
SQLX_OFFLINE=true cargo sqlx prepare -- --bin storyteller-web
|
||||
```
|
||||
|
|
@ -1,22 +0,0 @@
|
|||
# Local Kubernetes environment for standing up the APIs and their dependencies
|
||||
|
||||
(These are Paul's notes.)
|
||||
|
||||
- Install minikube, kubectl, docker, and musl from your OS package manager
|
||||
- `rustup target add x86_64-unknown-linux-musl`
|
||||
- `cargo build --release --bin storyteller-web --target=x86_64-unknown-linux-musl`
|
||||
- `minikube start --extra-config=apiserver.service-node-port-range=1-65535`
|
||||
- `eval $(minikube docker-env)`
|
||||
- `minikube mount /path/to/storyteller-rust/:/storyteller-rust &> /dev/null &`
|
||||
- `docker build -t storyteller-web -f localdev/Dockerfile .`
|
||||
- `kubectl apply -f localdev/kubernetes.yml`
|
||||
- Get the ip address of minikube with `minikube ip`, this is where all the services will be running on their
|
||||
regular ports (12345 for the api, 6379 for redis, 3306 for mysql)
|
||||
- That's it, we just spun up mysql, applied database migrations, spun up redis, and spun up our api
|
||||
|
||||
# Development tips
|
||||
- Rebuild and "deploy" (locally) with `cargo build --release --bin storyteller-web --target=x86_64-unknown-linux-musl && kubectl rollout restart storyteller-web`
|
||||
- View logs from the api with `kubectl logs storyteller-web<TAB>`
|
||||
- Delete everything minikube (if you start running out of disk space or what-have-you) with `minikube delete`
|
||||
- See pod status with `kubectl get pods`
|
||||
- If there are errors you can get more info with `kubectl describe pods`
|
||||
|
|
@ -1,56 +0,0 @@
|
|||
Local Development nginx proxy
|
||||
=============================
|
||||
|
||||
It isn't necessary to use nginx for development, but it can be nice. Nginx allows for a more
|
||||
sophisticated testing setup that makes cookie and CORS issues less of a pain.
|
||||
|
||||
You can use nginx to serve (locally-signed) SSL, redirect traffic to different ports, or run
|
||||
multiple copies of the app side by side.
|
||||
|
||||
### Hosts file
|
||||
|
||||
Set the following configs in your `/etc/hosts` file (as you would probably do even without nginx),
|
||||
|
||||
```
|
||||
127.0.0.1 dev.fakeyou.com
|
||||
127.0.0.1 api.dev.fakeyou.com
|
||||
127.0.0.1 devproxy.fakeyou.com
|
||||
|
||||
127.0.0.1 dev.storyteller.ai
|
||||
127.0.0.1 api.dev.storyteller.ai
|
||||
127.0.0.1 devproxy.storyteller.ai
|
||||
|
||||
# Deprecated development hostnames :
|
||||
127.0.0.1 jungle.horse
|
||||
127.0.0.1 api.jungle.horse
|
||||
```
|
||||
|
||||
### Nginx installation and administration
|
||||
|
||||
Configure Nginx per the checked in Nginx configs (and instructions) in
|
||||
the directory `/_develop/localdev/nginx-http-config`.
|
||||
|
||||
This should be straightforward, but if it isn't, contribute docs here.
|
||||
|
||||
### [Fix] Python 3.6 on Apple M1 Mac
|
||||
|
||||
Python3.6 isn't supported on Apple silicon, and it's not in homebrew. It can be installed with
|
||||
[nix using Rosetta](https://stackoverflow.com/a/65980989):
|
||||
|
||||
Download: https://nixos.org/download.html#nix-quick-install
|
||||
|
||||
```
|
||||
nix run nixpkgs.python36 -c python
|
||||
```
|
||||
|
||||
Install venv:
|
||||
|
||||
```
|
||||
nix run nixpkgs.python36 -c python -m venv python
|
||||
```
|
||||
|
||||
Install other packages on Mac that aren't used in venv:
|
||||
|
||||
```
|
||||
python3 -m pip install --user requests gdown youtube_dl
|
||||
```
|
||||
|
|
@ -1,36 +0,0 @@
|
|||
production builds and deployment
|
||||
================================
|
||||
|
||||
### Github Docker builds
|
||||
|
||||
The repository needs to be given read access to the base docker image:
|
||||
|
||||
https://github.com/orgs/storytold/packages/container/docker-base-images-rust-ssl/settings
|
||||
|
||||
These instructions assume running on GCP.
|
||||
|
||||
### Database migrations
|
||||
|
||||
1. Set `DATABASE_URL` in `.env` to the production secrets (DO NOT COMMIT!)
|
||||
2. Run `diesel migration run`
|
||||
|
||||
### Setting up public buckets without list permission
|
||||
|
||||
Public buckets that deny the `list` action should use the following Role:
|
||||
|
||||
`roles/storage.legacyObjectReader`
|
||||
|
||||
See:
|
||||
|
||||
* https://stackoverflow.com/a/56354633
|
||||
* https://cloud.google.com/storage/docs/access-control/making-data-public#buckets
|
||||
|
||||
### Generating Bucket Access Key and Secret Key
|
||||
|
||||
https://cloud.telestream.net/tutorials/how-to-setting-up-google-cloud-storage/
|
||||
|
||||
1. Go to the GCS page
|
||||
2. Click "settings"
|
||||
3. Click "interoperability" tab
|
||||
4. (enable interoperable access if not already set)
|
||||
5. Click "create new key"
|
||||
|
|
@ -1,17 +0,0 @@
|
|||
Production Database
|
||||
===================
|
||||
|
||||
Secrets can be subbed in for `.env` and the standard diesel migration tooling will work.
|
||||
|
||||
Dump a MySQL report
|
||||
-------------------
|
||||
|
||||
```bash
|
||||
mysql -u storyteller \
|
||||
-pPasswordHere \
|
||||
-h IP_HERE \
|
||||
-P 3306 \
|
||||
-D storyteller \
|
||||
-B \
|
||||
-e "select * from voice_clone_requests;" | sed "s/'/\'/;s/\t/\",\"/g;s/^/\"/;s/$/\"/;s/\n//g" > voice_clone_report.csv
|
||||
```
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
database schema
|
||||
================
|
||||
|
||||
(This is woefully incomplete and unmaintained.)
|
||||
|
||||
Twitch Model
|
||||
------------
|
||||
|
||||
- `twitch_user_id`
|
||||
Though numeric, this should always be a string per
|
||||
[https://discuss.dev.twitch.tv/t/bug-v5-api-returns--id-as-string-for-featured-channels/10310]
|
||||
and
|
||||
[https://discuss.dev.twitch.tv/t/type-of-user-id-in-api-responses/10205]
|
||||
|
||||
- `twitch_channel_id`
|
||||
This is the same as `twitch_user_id`. They're exactly 1:1. Prefer to
|
||||
use 'user_id' terminology.
|
||||
|
||||
- `twitch_username`
|
||||
Between 4 and 25 characters.
|
||||
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
redis schema
|
||||
============
|
||||
|
||||
Database 0: TTS / RateLimiter
|
||||
-----------------------------
|
||||
|
||||
TODO: Document rate limiter
|
||||
|
||||
Public statistics on usage displayed on the website:
|
||||
|
||||
* `ttsUseCount:{model_token}`: INCR, GET
|
||||
* `w2lUseCount:{template_token}`: INCR, GET
|
||||
|
||||
Temporary job progress reports to the user:
|
||||
|
||||
These are written as a single string, such as "downloading model", "uploading model",
|
||||
"generating preview", "executing", etc.
|
||||
|
||||
* `ttsDownloadExtraStatus:{inference_job_token}`: SETEX, GET
|
||||
* `ttsInferenceExtraStatus:{inference_job_token}`: SETEX, GET
|
||||
* `w2lDownloadExtraStatus:{inference_job_token}`: SETEX, GET
|
||||
* `w2lInferenceExtraStatus:{inference_job_token}`: SETEX, GET
|
||||
|
||||
Database 1: Obs / Twitch
|
||||
------------------------
|
||||
|
||||
* WIP
|
||||
|
|
@ -1,203 +0,0 @@
|
|||
# PageDraw — Visual Design & Architecture Review
|
||||
|
||||
**Surface:** `@storyteller/ui-pagedraw` image editor + its `artcraft-webapp` integration (`/edit-image`).
|
||||
**Lens:** Awwwards / Linear-tier "premium app chrome" rubric, adapted for an in-product creative tool.
|
||||
**Date:** 2026-06-15 · **Branch:** `worktree-pagedraw-webapp`
|
||||
|
||||
---
|
||||
|
||||
## 0. How to read this
|
||||
|
||||
PageDraw is a **professional canvas tool**, not a marketing page. So the elite-design rubric is
|
||||
applied in its *app-chrome* register — the north star is **Figma / Linear / Photoshop-web**, not an
|
||||
Awwwards landing page. That means:
|
||||
|
||||
- **Applies:** token discipline, concentric radii, hairline + inset-highlight surfaces ("double-bezel"),
|
||||
island/pill controls, custom-eased micro-motion, z-index hygiene, premium type ramp.
|
||||
- **Does _not_ apply:** `py-40` macro-whitespace, hero typography, editorial-split layouts, scroll-reveal
|
||||
entrances. A tool surface is dense and persistent by design; forcing landing-page whitespace would hurt it.
|
||||
|
||||
Citations are `file:line` against the current tree.
|
||||
|
||||
---
|
||||
|
||||
## 1. Verdict
|
||||
|
||||
| Axis | Score | One-liner |
|
||||
|---|---|---|
|
||||
| Color system | ⚠️ 5/10 | Good token foundation, but hardcoded greys (`#1b1b1b`, `#303030`, `#404040`) leak through a *themeable lib*. |
|
||||
| Typography | ❌ 3/10 | No type ramp, no font-family ownership; rides whatever the host sets. Banned-by-rubric default stack risk. |
|
||||
| Radii | ⚠️ 4/10 | Five radius scales in play (`sm`→`2xl`→`full`) with no concentric nesting. |
|
||||
| Surfaces (border/shadow) | ⚠️ 5/10 | Flat `shadow-lg` everywhere; no inset highlights; no double-bezel. Reads "functional", not "machined". |
|
||||
| Motion | ❌ 3/10 | Only `transition-colors`/`ease-in-out`, durations 75–200ms; **zero custom easing**, no spring, no entrance choreography. |
|
||||
| Layout / z-index | ⚠️ 6/10 | Clean fixed-panel composition, but a `z-[9999999]` escape hatch and a 6-tier ad-hoc z-stack. |
|
||||
| Icons | ✅ 7/10 | FontAwesome Pro (solid + regular) — acceptable, but solid weight is heavier than the rubric's "ultra-light line" ideal. |
|
||||
| Architecture | ⚠️ 5/10 | Clean adapter boundary; but `PaintSurface.tsx` (2005 LOC), `SceneState.ts` (1539), `PageDraw.tsx` (1292) are oversized, and the lib hardcodes host-theme colors. |
|
||||
|
||||
**Bottom line:** the editor is *structurally competent and functional* but visually reads as
|
||||
"capable internal tool," not "$150k product surface." The single highest-leverage fix is a **shared
|
||||
design-token layer** (color + radius + motion + type), because today the lib both (a) reaches for
|
||||
host tokens (`bg-ui-panel`, `text-base-fg`) *and* (b) hardcodes its own greys — so it can never be
|
||||
fully reskinned by the webapp vs. the desktop app.
|
||||
|
||||
---
|
||||
|
||||
## 2. What's already good (keep)
|
||||
|
||||
- **Token-aware base.** Most surfaces use semantic tokens (`bg-ui-panel`, `bg-ui-controls`,
|
||||
`text-base-fg`, `border-ui-panel-border`) — the right instinct (SideToolbar, PromptEditor, HistoryStack).
|
||||
- **Clean platform boundary.** The lib has zero REST/Tauri imports; everything I/O flows through
|
||||
`PageDrawAdapter` (`adapter.ts`). The webapp adapter (`web-adapter.tsx`) and desktop adapter stay
|
||||
swappable. This is genuinely good architecture and should be protected.
|
||||
- **Performance-minded canvas.** Composite pre-baking on idle (`PageDraw.tsx` ~661–710) and worker
|
||||
encoding (`generatePipeline.worker.ts`) keep the generate click off the main thread. Don't regress this.
|
||||
- **CORS already handled** for browser canvas export (`crossOrigin="anonymous"` in `Node.ts:134`,
|
||||
`SceneState.ts:1473`, `imageHelpers.ts:16`) — the web port works because of this.
|
||||
|
||||
---
|
||||
|
||||
## 3. Findings & recommendations
|
||||
|
||||
### 3.1 Color — stop hardcoding greys in a themeable lib ⚠️ P0
|
||||
**Current:** semantic tokens *and* raw hex coexist:
|
||||
- `#1b1b1b` pegboard (`pagedraw.css:6`), `#1A1A1A`/`#333333` context menu (`ContextMenu.tsx:44`),
|
||||
`#303030`/`#404040` dividers + popovers (`SideToolbar.tsx:381,550`), `bg-zinc-700/600` swatch
|
||||
(`SideToolbar.tsx:173`), plus ad-hoc `bg-blue-600/500` accents (`PageDraw.tsx:157`, base selector).
|
||||
- Accent color is inconsistent: the lib's primary action uses raw **blue-600**, while the webapp's
|
||||
design system uses `--primary` / `brand-primary`. So the editor's blue doesn't match the product.
|
||||
|
||||
**Why it fails the bar:** a premium component library exposes *one* surface ramp and *one* accent, both
|
||||
driven by tokens, so it reskins per host. Hardcoded greys mean the desktop and web themes can drift, and
|
||||
the raw `blue-600` clashes with `brand-primary` used elsewhere in the webapp.
|
||||
|
||||
**Do:**
|
||||
- Replace every raw grey with the existing ramp: `#1b1b1b`→`bg-ui-background`, `#1A1A1A`→`bg-ui-panel`,
|
||||
`#303030`/`#404040`→`border-ui-panel-border` / `bg-ui-controls`. For the pegboard, add a token
|
||||
`--st-canvas` so hosts can theme the canvas backdrop.
|
||||
- Replace `blue-600/500` accents with `bg-primary` / `text-primary` (the token the rest of the webapp uses),
|
||||
so the Generate/Edit-3D buttons match `brand-primary`.
|
||||
- Move the react-colorful overrides off pure `rgb(255,255,255)` borders (`pagedraw.css:15,19`) onto
|
||||
`white/15` hairlines for consistency with the rest of the chrome.
|
||||
|
||||
### 3.2 Typography — own a type ramp ❌ P0
|
||||
**Current:** no `font-family` anywhere in the lib (grep: 0 hits); only `text-xs/sm/base/lg/xl` +
|
||||
`font-medium/semibold/bold`. The editor inherits whatever the host's body font is.
|
||||
|
||||
**Why it fails the bar:** the rubric explicitly bans the default web stack (Inter/Roboto/Arial/Helvetica).
|
||||
Inheriting an unspecified stack means the tool can render in a banned font with no recourse, and there's
|
||||
no deliberate hierarchy (numerics in sliders, labels, modal titles all share one undifferentiated ramp).
|
||||
|
||||
**Do:**
|
||||
- Adopt the product's display/grotesk token (e.g. a `font-sans` mapped to Geist / Plus Jakarta Sans at
|
||||
the app level) and reference it in the lib's root container, not per-component.
|
||||
- Define a small semantic ramp: `text-[11px] tracking-[0.16em] uppercase` for toolbar/section labels,
|
||||
`text-sm` for controls, `text-base font-semibold` for modal titles. Apply `tabular-nums` to slider
|
||||
read-outs (`SliderWithIndicator.tsx`) and dimension fields (`BlankCanvasModal`) so numbers don't jitter.
|
||||
|
||||
### 3.3 Radii — collapse to a concentric scale ⚠️ P1
|
||||
**Current:** `rounded-sm / md / lg / xl / 2xl / full` all appear, with no parent→child relationship
|
||||
(e.g. `rounded-2xl` card at `PageDraw.tsx:1123`, `rounded-xl` panels in SideToolbar, `rounded-lg`
|
||||
buttons in InpaintToolBar).
|
||||
|
||||
**Do:** pick a 3-step system — **`rounded-2xl` shells, `rounded-xl` inner cards, `rounded-lg` controls,
|
||||
`rounded-full` pills** — and apply concentrically (the "double-bezel" curve math: inner radius =
|
||||
outer − padding). Drop `rounded-sm`/`rounded-md` from the editor entirely.
|
||||
|
||||
### 3.4 Surfaces — add the "machined" double-bezel ⚠️ P1
|
||||
**Current:** panels are a single `bg-ui-panel` + `border` + `shadow-lg`. No nested enclosure, no inset
|
||||
top-highlight. Reads flat.
|
||||
|
||||
**Do (for the four floating panels — SideToolbar, HistoryStack, PromptEditor, InpaintToolBar):**
|
||||
- Outer shell: `bg-ui-panel/80 ring-1 ring-white/10 backdrop-blur-xl p-1.5 rounded-2xl`.
|
||||
- Inner core: own bg + `shadow-[inset_0_1px_0_rgba(255,255,255,0.08)] rounded-xl`.
|
||||
- This is the single change that most moves the surface from "tool" to "premium hardware." The panels are
|
||||
already `fixed` (`PageDraw.tsx:1144,1175,1195`), so `backdrop-blur` here is GPU-safe per the perf rules.
|
||||
|
||||
### 3.5 Motion — replace linear/ease with spring-eased micro-motion ❌ P0 (cheap, high impact)
|
||||
**Current:** `transition-colors`/`transition-all` with `ease-in-out` and `duration-75/150/200`
|
||||
(`SideToolbar.tsx:381,550`, `SliderWithIndicator.tsx:65`, `ContextMenu.tsx:53`). No custom easing,
|
||||
no entrance/exit on popovers, color pickers, or the context menu (they pop instantly).
|
||||
|
||||
**Why it fails the bar:** instant state changes and `ease-in-out` are explicitly banned. A pro tool still
|
||||
deserves *interpolated* motion — it's what separates Figma from a jQuery plugin.
|
||||
|
||||
**Do:**
|
||||
- Standardize on one curve: `transition-[transform,opacity,background-color] duration-300
|
||||
ease-[cubic-bezier(0.32,0.72,0,1)]`.
|
||||
- Tool buttons: `active:scale-[0.97]` press physics; selected tool gets a spring-eased indicator slide
|
||||
rather than a hard background swap.
|
||||
- Popovers / color picker / context menu: animate in with `opacity-0 translate-y-1 scale-[0.98]` →
|
||||
resolved, ~180ms. Transform+opacity only (perf rule).
|
||||
- History tiles: stagger their `opacity` reveal when a generation lands (they already use
|
||||
`transition-opacity`, just add per-index delay).
|
||||
|
||||
### 3.6 Layout & z-index — tame the stack ⚠️ P1
|
||||
**Current:** clean fixed-panel composition, but z-index is ad-hoc across 6 tiers and includes a
|
||||
`z-[9999999]` on the context menu (`ContextMenu.tsx:44`) and a one-off `z-[60]` (`PageDraw.tsx:237`).
|
||||
|
||||
**Do:** define a named scale in one place — `canvas:0, chrome:10, floating-controls:20, overlay-button:40,
|
||||
menu:50` — and replace the magic numbers. The portal menu only needs to clear the chrome, not `9999999`.
|
||||
|
||||
### 3.7 Responsive — declare intent ⚠️ P2
|
||||
**Current:** essentially desktop-only; fixed sidebars assume a wide viewport; only LightboxDetails has
|
||||
`sm:` variants. The webapp wrapper scopes the lib correctly (`pagedraw.tsx` `translateZ(0)` containing
|
||||
block) but there's no mobile story.
|
||||
|
||||
**Do:** mirror the Edit-3D page's explicit **mobile gate** (`pagescene.tsx` `MobileGate`) on `/edit-image`
|
||||
rather than letting fixed toolbars overlap on touch. A precision canvas tool is legitimately desktop-first;
|
||||
just say so deliberately instead of degrading silently.
|
||||
|
||||
---
|
||||
|
||||
## 4. Architecture notes
|
||||
|
||||
- **Oversized modules.** `PaintSurface.tsx` (2005 LOC), `SceneState.ts` (1539), `PageDraw.tsx` (1292).
|
||||
These are review/maintenance hotspots. Suggest extracting from `SceneState.ts`: history/undo slice,
|
||||
base-image-load slice, and generation/pending slice into composable Zustand slices. `PageDraw.tsx`'s
|
||||
generate-flow (cases 1–3, ~780–868) could move to a `useGenerate()` hook.
|
||||
- **Theme ownership.** Per §3.1, the lib should *only* consume tokens. Today it mixes tokens + hex, which
|
||||
defeats the adapter pattern's promise (one component, many themed hosts).
|
||||
- **Adapter surface is healthy.** `PageDrawAdapter` is small and intentional; the web vs. Tauri split is
|
||||
clean. The webapp's `enqueueInpaint` stub and 402→credits-modal handling are explicit and documented.
|
||||
- **Icon weight.** FontAwesome **solid** is the heaviest option; if a line/light set is available
|
||||
(Phosphor/Remix or FA light), the toolbar would read more refined. Low priority.
|
||||
|
||||
---
|
||||
|
||||
## 5. Prioritized roadmap
|
||||
|
||||
| P | Item | Effort | Payoff |
|
||||
|---|---|---|---|
|
||||
| **P0** | Token sweep: kill hardcoded greys; unify accent on `--primary` (§3.1) | M | Highest — enables true theming + product-consistent accent |
|
||||
| **P0** | Own a font-family + semantic type ramp at the lib root (§3.2) | S | High — removes banned-font risk, adds hierarchy |
|
||||
| **P0** | One custom easing curve + press/popover micro-motion (§3.5) | S | High — cheapest "feels expensive" win |
|
||||
| **P1** | Double-bezel + inset highlights on the 4 floating panels (§3.4) | M | High — "machined" surface upgrade |
|
||||
| **P1** | Concentric 3-step radius system (§3.3) | S | Medium — visual coherence |
|
||||
| **P1** | Named z-index scale; remove `z-[9999999]` (§3.6) | S | Medium — maintainability |
|
||||
| **P2** | Explicit mobile gate on `/edit-image` (§3.7) | S | Medium — correctness on touch |
|
||||
| **P2** | Split `SceneState.ts` / `PaintSurface.tsx` into slices (§4) | L | Medium — maintainability |
|
||||
|
||||
---
|
||||
|
||||
## 6. Appendix — proposed token additions
|
||||
|
||||
```css
|
||||
/* canvas + chrome surface ramp (lib should consume only these) */
|
||||
--st-canvas: <theme>; /* replaces #1b1b1b pegboard */
|
||||
--st-accent: var(--primary); /* replaces blue-600/500 */
|
||||
--st-hairline: rgb(255 255 255 / 0.10);
|
||||
--st-inset-hi: inset 0 1px 0 rgb(255 255 255 / 0.08);
|
||||
|
||||
/* motion */
|
||||
--st-ease: cubic-bezier(0.32, 0.72, 0, 1);
|
||||
--st-dur-fast: 180ms;
|
||||
--st-dur: 300ms;
|
||||
|
||||
/* radius scale (concentric) */
|
||||
--st-r-shell: 1.25rem; /* rounded-2xl */
|
||||
--st-r-card: 0.875rem; /* rounded-xl */
|
||||
--st-r-control: 0.5rem; /* rounded-lg */
|
||||
```
|
||||
|
||||
> Scope note: every recommendation here is **styling/architecture only** — none change the generate,
|
||||
> polling, upload, bg-removal, or adapter contracts validated during the functional debug pass.
|
||||
|
|
@ -1,227 +0,0 @@
|
|||
# PageScene Animations — Branch Handoff
|
||||
|
||||
Branch: `frontend/pagescene-animations` (forked from `main` @ `6329045350`, 2026-08-04).
|
||||
Everything below is TypeScript-typechecked per-lib (`tsc -p <lib>/tsconfig.lib.json`); runtime
|
||||
verification status is tracked in the checklist at the bottom. Drag-and-drop has had one manual
|
||||
pass (confirmed working); most other surfaces have not been launched.
|
||||
|
||||
## What this branch does
|
||||
|
||||
Makes skeletal animations a first-class asset in the 3D editor (pagescene): browse them in the
|
||||
asset drawer, drag them onto characters or the timeline with live rig-compatibility feedback,
|
||||
play clips baked into uploaded models, preview animations everywhere models are previewed, and
|
||||
upload Mixamo FBX files (with or without mesh) end to end.
|
||||
|
||||
## Commit map
|
||||
|
||||
| Commit | What |
|
||||
|--------------|-------------------------------------------------------------------------|
|
||||
| `7af3fb9856` | Animations tab in AssetModal; AnimationsDrawer retired |
|
||||
| `1fcbc55780` | Pointer drag-and-drop with live rig-compatibility badge |
|
||||
| `79d7ade8bc` | Baked-in clips on the timeline (+ clip-row eligibility widened) |
|
||||
| `142279414e` | Animation dropdown in Viewer3D + upload preview |
|
||||
| `913905308e` | Clip fixes: vanishing character, 1 s strips, select-to-delete, undo |
|
||||
| `60251104e3` | FBX accepted at the picker, converted to GLB in-browser |
|
||||
| `0942ec4804` | Nx tsconfig references for the new ui-select dependency |
|
||||
| `d0c7b65a6e` | FBX conversion moved into a Web Worker (main-thread fallback) |
|
||||
| `8e51b6c524` | Explicit "Upload as Animation" toggle + required duration_millis |
|
||||
| `00ebeae10c` | User's uploaded animations merged into the Animations tab |
|
||||
| `e1c7a0c7b0` | Persistent skeleton-helper toggle in the outliner |
|
||||
| `30f40c0ddb` | Previewer bone toggles + mesh-less framing/mixer-binding fixes |
|
||||
| `5f7ba7bbc5` | AnimationsModal (All/Presets/Uploaded) + timeline quality-of-life |
|
||||
| `d55735620a` | Timeline ruler tick marks + strip button spacing |
|
||||
| `79875d8c76` | Upload animations from the AnimationsModal (initialCategory) |
|
||||
| *(this)* | NodeHierarchyHelper: bone toggle works for converted mesh-less GLBs |
|
||||
|
||||
## Features
|
||||
|
||||
### 1. Animations tab in the asset drawer (`AssetModal`)
|
||||
|
||||
- New left-rail section fed by `GET /v1/media_files/list_featured?filter_engine_categories=animation`
|
||||
(GLB/GLTF only — the clip loader is GLTF-based). Featured results REPLACE the 37 hardcoded
|
||||
`demoAnimationItems`; demos are only the empty-API fallback.
|
||||
- The session user's own uploaded animations (`list/user`, same filter) render AHEAD of the
|
||||
featured set. Fetched on every modal open and after each upload; fails silently for anonymous
|
||||
users (`suppressErrorToast` on the user-media fetch path).
|
||||
- Card click appends the clip to the selected character's row at the earliest free slot (toast
|
||||
when no character is selected or the row is full). The old right-docked `AnimationsDrawer` is
|
||||
deleted; the old HTML5-DnD path (`ANIMATION_CLIP_MIME`) is gone with it.
|
||||
|
||||
### 2. Drag-and-drop with rig-compatibility badge (`DndAsset`)
|
||||
|
||||
- Animation cards ride the same pointer-drag pipeline as every other asset. On drag start the
|
||||
clip's track node names are preloaded (`loadRawGlb`, cached per media id); hovering a
|
||||
character (or any skinned object) checks those names against its nodes — the exact direct-bind
|
||||
rule `CharacterAnimationManager` plays by. No retargeting (explicitly out of scope).
|
||||
- The shared `DragGhost` shows the verdict: green check (binds) / red slash (no target or rig
|
||||
mismatch); nothing while the clip is still loading. Incompatible or slot-less drops are
|
||||
rejected with a toast.
|
||||
- Drop targets: a character in the 3D scene (earliest free slot) or a timeline row
|
||||
(`data-clip-drop-uuid`; pointer-x → time via the `data-timeline-ruler` rect).
|
||||
|
||||
### 3. Timeline clips: behavior + UX
|
||||
|
||||
- **Strips default to 1 s** and the strip width is the authoritative play window
|
||||
(`LaneRuntime.stripDuration`): trimming genuinely cuts playback; loop repeats inside the
|
||||
strip; a non-loop strip wider than its clip freezes on the last frame. On load a strip only
|
||||
shrinks (clip shorter than default) — never grows.
|
||||
- **Rest-pose in gaps**: `Skeleton.pose()` was replaced with a rest-pose snapshot captured at
|
||||
mixer creation (all nodes BELOW the object root; root excluded so gizmo/keyframes never snap
|
||||
back). `pose()` rebuilds bone locals from inverse bind matrices and mis-scales cm-rigs — that
|
||||
was the "character vanishes when moving a strip" bug.
|
||||
- **Selection UX**: strips are click-to-select; × renders only on the selected strip;
|
||||
Del/Backspace deletes; removal is individually undoable (`RemoveClipLaneAction` — the one
|
||||
exception to clip edits riding the timeline Save/Cancel session).
|
||||
- **Full rows reject adds**: `resolveFreeStart` returns null instead of accepting an
|
||||
overlapping clamp.
|
||||
|
||||
### 4. Baked-in clips
|
||||
|
||||
- `ClipStrip.bakedClipIndex` marks a strip sourced from the object's own `animations[]`
|
||||
(resolved synchronously — no media fetch). Objects with baked clips get a timeline clip row
|
||||
with a "+" picker listing them.
|
||||
- Clip-row eligibility is wider than characters: any skinned object (creatures, rigged uploads)
|
||||
accepts animation drags (bind check gates); baked-only objects get a row for the picker.
|
||||
- INVARIANT: baked clips are never removed from the THREE model. Removing a baked strip only
|
||||
unschedules it; the clip stays on `object.animations` and in the picker. Library strips stay
|
||||
fully removable.
|
||||
- Outliner metadata (`hasSkeleton`, `bakedClips`) is computed in `convert_object` and refreshed
|
||||
after every GLB load (`SceneDeps.onGlbLoaded` → `refreshOutliner`).
|
||||
|
||||
### 5. Previewers (Viewer3D + upload preview)
|
||||
|
||||
- Both play baked clips: autoplay the first, dropdown with every clip + "T-pose (none)", hidden
|
||||
entirely for animation-less models.
|
||||
- Both have a bone icon toggling a skeleton overlay — default ON when the model has no mesh
|
||||
(Mixamo "without skin" exports), so animation-only files are visible. The upload thumbnail
|
||||
snapshot then captures the skeleton render.
|
||||
- The overlay adapts to the rig kind: real `THREE.Bone` joints get `THREE.SkeletonHelper`;
|
||||
mesh-less models whose joints re-imported as plain nodes get `NodeHierarchyHelper`
|
||||
(exported from `viewer-3d`) — GLTF only round-trips bone-ness through a *skin*, and a
|
||||
mesh-less export has no skin, so converted FBX animations lose `Bone`-ness entirely. The
|
||||
toggle shows for `hasBones || !hasMesh`.
|
||||
- Latent bugs fixed: upload preview discarded `gltf.animations`; its child re-parenting loop
|
||||
skipped every other child (`scene.add` mutates the array mid-forEach); its mixer bound before
|
||||
the model entered the scene (silently played nothing); camera framing NaN'd on empty
|
||||
bounding boxes (now falls back to bone world positions); camera now fits once, not per-child.
|
||||
|
||||
### 6. FBX upload support
|
||||
|
||||
- The 3D uploader accepts `.fbx` and normalizes it at the door: `FBXLoader` → `GLTFExporter`
|
||||
→ binary GLB with clips preserved. Runs in a dedicated Web Worker (transferred buffers, jobs
|
||||
naturally serialized); a broken worker environment falls back to the main thread. The raw FBX
|
||||
can never reach the backend — retry of a failed conversion re-converts.
|
||||
- Caveats: embedded textures may not survive; units are exported as parsed (cm-scaled Mixamo
|
||||
rigs are not rescaled).
|
||||
|
||||
### 7. Upload as Animation
|
||||
|
||||
- Mutually exclusive checkbox next to "Upload as Character" → `engine_category=animation`.
|
||||
Mesh-less files preselect it (preview loader reports `hasMesh:false`); a user click always
|
||||
wins. Rig-type dropdown defaults to Mixamo (characters keep Mixamo ArKit first).
|
||||
- The backend-required `maybe_duration_millis` is read from the longest clip at submit time;
|
||||
clip-less files are flagged and excluded client-side.
|
||||
|
||||
### 8. Outliner skeleton toggle (`SkeletonHelperController`)
|
||||
|
||||
- Bone icon on skinned rows toggles a persistent `SkeletonHelper` on LAYER 1 (editor-only —
|
||||
the render camera renders layer 0, so skeletons never appear in captures/recordings).
|
||||
- Persists as `userData.skeletonVisible` through scene save/load (explicit restore in the proxy
|
||||
load path). Reconciliation rides `OutlinerRefreshedEvent` — add/delete/load/new-scene and the
|
||||
post-GLB-load refresh all funnel there.
|
||||
- FK/pose-mode compatible: `PoseModeChangedEvent("pose")` suppresses the persistent helper for
|
||||
the posed character (FKHelper draws its own rig); exit restores it. The toggle state itself
|
||||
is never touched.
|
||||
|
||||
## Manual test checklist (outstanding)
|
||||
|
||||
- [ ] Prod data: does `list_featured?filter_engine_categories=animation` return rows? If yes
|
||||
they replace the demo clips in the tab; if empty, the 37 demos show.
|
||||
- [ ] Clip→rig bind smoke test on shipped characters (runtime-unverified `mixamorig:*`
|
||||
assumption; a "bound 0 tracks" console warning means retargeting becomes the real task).
|
||||
- [ ] Creature experiment: drag a clip onto a creature — badge should reflect bind result.
|
||||
- [ ] Baked clips: "+" picker on an animated object's row; add/remove/re-add; save/reload
|
||||
persistence (the `object.animations` stash on load is gated on `auto_add` — statically
|
||||
unconfirmed for every load path).
|
||||
- [ ] Clip UX round: strips at 1 s; trim semantics; select → ×/Del; undo/redo of removal;
|
||||
character stays visible when a strip moves off the playhead.
|
||||
- [ ] FBX round trip: pick `.fbx` → converts (worker; check console for worker load failures —
|
||||
falls back to main thread) → skeleton visible + clip dropdown → Upload as Animation
|
||||
preselected → appears at front of Animations tab → drag onto character → plays.
|
||||
- [ ] Converted-FBX scale: if placed objects come in ~100× too big, the converter needs a
|
||||
normalize-scale step.
|
||||
- [ ] Outliner bone toggle: on/off, FK-mode enter/exit restore, save/reload persistence, and
|
||||
absence from Capture/Record output.
|
||||
- [ ] Previewer dropdown + bone toggle across lightbox, media page, lightbox-modal,
|
||||
ImageTo3DExperience, upload modal.
|
||||
|
||||
## Debug pass + hardening round (post-review)
|
||||
|
||||
A four-agent review of the combined branch produced 30 findings (5 high / 12 medium / 13 low);
|
||||
ALL have been fixed or explicitly closed, one commit per finding (range `72cb218e..b21732d1`).
|
||||
Highlights, roughly in commit order:
|
||||
|
||||
- Highs: baked clips cloned per lane (shared-AnimationAction playback/removal corruption);
|
||||
runtime revalidation for objects recreated under the same uuid (delete→undo, reloads) — later
|
||||
hardened into LOOP-FREE terminal lane states (bound / missing / unbindable, reassessed only on
|
||||
root-instance change; the pre-fetch rig gate is sound because every load path stamps the target
|
||||
uuid only post-load); skeleton-helper sync self-heals across in-session reloads; upload modal
|
||||
can no longer report success for empty/partial submissions; ref-guarded double-submit.
|
||||
- Mediums: strip removal mirrored into the Save/Cancel `saved` snapshot (Cancel can't resurrect,
|
||||
no phantom undo); stale strip/keyframe selections pruned in the bridge + live-checked before
|
||||
swallowing Del/Backspace; one winning lane per character at flush boundaries (blend-glitch
|
||||
frame was being ENCODED into recordings); orphaned tracks/lanes filtered from scene JSON at
|
||||
save; per-clip queued-drop tokens; animation raycasts gated to the canvas element + honest
|
||||
front-most-surface occlusion; virtualized-grid thumbnail state keyed by item; one shared
|
||||
lazily-fetched animation-library store (was double-fetch + double-toast + divergent lists);
|
||||
previewer rest-pose snapshots replace `Skeleton.pose()` everywhere; all-node camera-framing
|
||||
fallback for plain-node rigs; upload-preview races (superseded-load cancellation, snapshot
|
||||
after helper creation, File-identity row statuses, retry re-entry).
|
||||
- Lows: duration-shrink re-fits strips; baked-clip picker portals out of the scrolling row list;
|
||||
timeline resets on scene switch/new scene instead of leaking across; click-to-add accepts every
|
||||
drag-eligible target; queued drops bail on a torn-down editor (`Editor.isLive`); top-level root
|
||||
lookups unified; flick-click race killed via `wasDragGesture`; converting-overlay control reset;
|
||||
conversion completions reload only the previewed entry; worker `onmessageerror` recovery +
|
||||
corrupt-vs-clip-less error messages + explicit `initialCategory` wins over mesh-less
|
||||
auto-detect; Uploaded tab pages past the 100-item cap.
|
||||
- Closed without code: the "compat cache stales on streaming characters" finding (invalidated —
|
||||
uuid-resolvable roots are always fully loaded) and the Tauri `promptSignup` gap (desktop is
|
||||
always signed in by adapter contract).
|
||||
|
||||
## Landed since the original handoff
|
||||
|
||||
- **AnimationsModal** (was deferred — now built, `5f7ba7bbc5`/`79875d8c76`): standalone modal
|
||||
with All / Presets / Uploaded tabs, opened from an "Add Animation" button next to Enter Pose
|
||||
Mode and from the "+" add-asset menu. Adding a clip expands the timeline and scrolls to the
|
||||
character (`timelineRevealObjectUuid`); the modal closes after an add unless "Reopen after
|
||||
adding" is on. Its "Upload animation" button opens `UploadModal3D` with
|
||||
`initialCategory="animation"` (anonymous users get the signup prompt), landing on the
|
||||
Uploaded tab afterwards. `AssetModal`'s animations tab was folded into this.
|
||||
- **Timeline QoL**: ruler tick marks (major/second/sub-tick), selected-object-only track
|
||||
filter, directional expand/collapse chevrons, thumbnail-less animation cards get an icon
|
||||
placeholder.
|
||||
|
||||
## Deferred (explicitly out of scope for this branch)
|
||||
|
||||
- **Split-screen preview window** (based on the `?output=` `DemoOutputOverlay`) — parked until
|
||||
explicit approval.
|
||||
- **Retargeting** (`SkeletonUtils.retargetClip`) — too complex for now; mismatched rigs are
|
||||
simply rejected by the bind check.
|
||||
- **Clip blending/layering** — one non-overlapping row per object, by design.
|
||||
|
||||
## Key files
|
||||
|
||||
| Area | Files |
|
||||
|---------------------|-------------------------------------------------------------------------------------------|
|
||||
| Drawer tab | `pagescene/src/lib/comps/AssetMenu/AssetModal.tsx`, `shared/ItemElement.tsx` |
|
||||
| Drag-and-drop | `pagescene/src/lib/DragAndDrop/DndAsset.ts`, `gallery-modal/src/lib/DragGhost.tsx` |
|
||||
| Timeline data/engine| `pagescene/src/lib/engine/timeline/types.ts`, `engine/editor/TimelineController.ts`, |
|
||||
| | `engine/animation/CharacterAnimationManager.ts`, `engine/editor/actions/RemoveClipLaneAction.ts` |
|
||||
| Timeline UI | `pagescene/src/lib/comps/Timeline/TimelineEditor.tsx`, `TimelineClipRow.tsx` |
|
||||
| Skeleton toggle | `pagescene/src/lib/engine/editor/SkeletonHelperController.ts`, `comps/Outliner/Outliner.tsx` |
|
||||
| Previewers | `viewer-3d/src/lib/viewer-3d.tsx`, `upload-modal/.../utilities/loadPreviewOnCanvas.ts` |
|
||||
| FBX + upload | `upload-modal/.../utilities/convertFbxToGlb.ts` (+ `.worker.ts`), `UploadModal3D.tsx`, |
|
||||
| | `UploadFiles3D.tsx`, `utilities/readGlbAnimationDuration.ts` |
|
||||
|
||||
The pagescene `CLAUDE.md` "Mixamo animation clips" section has been kept current with these
|
||||
changes and carries the same invariants in engine-doc form.
|
||||
|
|
@ -1,227 +0,0 @@
|
|||
# Artcraft Moodboard — Design & Flow
|
||||
|
||||
> How the moodboard looks, feels, and flows. Written through a high-end product-design lens
|
||||
> (Linear/Apple-tier restraint: haptic depth, spring-physics motion, calm surfaces) and grounded
|
||||
> in the shipped implementation (`@storyteller/ui-moodboard`). Companion to `DESIGN_SYSTEM.md`
|
||||
> (tokens/components), `FEATURE_PLAN.md` (roadmap), and `FEATURE_BACKLOG.md` (what's next).
|
||||
|
||||
---
|
||||
|
||||
## 1. Design philosophy
|
||||
|
||||
Three convictions drive every decision:
|
||||
|
||||
1. **The board is an input to generation, not a graveyard.** Generic moodboards (Pinterest,
|
||||
Milanote) are dead ends — references go in, nothing comes out. Artcraft owns the engine, so the
|
||||
board is one gesture away from a generation and vice-versa. Every surface keeps that loop in
|
||||
reach without ever shouting about it.
|
||||
2. **Native, then elevated.** The moodboard lives *inside* Artcraft, so it speaks the app's visual
|
||||
language (the `--st-*` theme tokens, Fira Sans, the `glass` material, the brand blue). Elevation
|
||||
comes from *composition* — nested "double-bezel" enclosures, floating glass islands, motion with
|
||||
mass — never from a louder palette. The board should feel like it was always part of the app.
|
||||
3. **The imagery is the interface.** Chrome recedes; references dominate. No like-counts, no
|
||||
notification noise, no decorative gradients competing with the user's work. Calm by default
|
||||
(the Cosmos/Are.na lesson), dense where it counts.
|
||||
|
||||
The felt quality we're after: **a precise, quiet instrument** — fast, tactile, and a little bit
|
||||
joyful, where heavy operations feel weightless and nothing ever teleports.
|
||||
|
||||
---
|
||||
|
||||
## 2. Visual language (the short version)
|
||||
|
||||
The full token/component spec is in `DESIGN_SYSTEM.md`; the design *intent* behind it:
|
||||
|
||||
- **Double-bezel enclosures.** Cards, the inspector, and toolbars are never flat on the canvas.
|
||||
A subtle outer shell (plane-1, hairline ring, `rounded-[18px]`, `p-1`) cradles an inner core
|
||||
(plane-2, concentric `rounded-[14px]`, an inset top-highlight). It reads like a glass plate
|
||||
seated in a machined tray — physical, considered, expensive.
|
||||
- **Floating glass islands.** The toolbar and the view switch are detached pills, not edge-glued
|
||||
bars (`glass` + hairline + ambient shadow, `backdrop-blur` only on these fixed elements — never
|
||||
on the scrolling grid, where blur would torch scroll performance).
|
||||
- **Spring-physics motion.** Custom cubic-beziers, never `linear`/`ease-in-out`. A hover lift, a
|
||||
card landing on drop with a touch of overshoot, a palette fading up. Animate only `transform`,
|
||||
`opacity`, `filter` — GPU-safe. `prefers-reduced-motion` collapses springs to fades.
|
||||
- **Concentric squircle radii** and **ambient, layered shadows** (never a harsh `shadow-md`).
|
||||
- **One display moment.** Body/UI is Fira Sans (the app font, for zero context-switch); the only
|
||||
"display" flourish is the editorial empty-state heading and presentation mode.
|
||||
|
||||
---
|
||||
|
||||
## 3. Information architecture
|
||||
|
||||
```
|
||||
Board (durable model, localStorage today → backend later)
|
||||
├── items: image · video · text · link · color (each: tags, rating, deterministic aspect)
|
||||
├── sections (model-ready; UI is a near-term add)
|
||||
└── rendered three ways — ALL views are projections of the same board:
|
||||
├── GRID virtualized masonry — the default; scan / search / triage at volume
|
||||
├── CANVAS freeform Konva stage — arrange / compare / plan spatially
|
||||
└── PRESENT full-bleed deck — review & share
|
||||
```
|
||||
|
||||
**Why grid-first.** 90% of moodboard time is scanning, searching, and triaging — the grid wins
|
||||
those high-frequency jobs and stays at 60fps (canvas-only tools like Milanote lag at a few hundred
|
||||
items; we virtualize from item aspect ratios so only the viewport mounts). The freeform canvas is
|
||||
the *delighter* for spatial thinking, not the foundation. **The board, not the view, is what plugs
|
||||
into generation** — "use as reference" works regardless of how you're looking at it.
|
||||
|
||||
---
|
||||
|
||||
## 4. Core flows
|
||||
|
||||
Each flow below lists the **gesture**, then the **design rationale**.
|
||||
|
||||
### 4.1 Capture — get references in with zero friction
|
||||
The fastest path wins, so we offer every path and make the empty state *say so*.
|
||||
|
||||
- **Drag images** from the desktop onto the board → a dashed primary-tinted drop overlay confirms
|
||||
the target; files land at the cursor.
|
||||
- **Paste** — an image from the clipboard drops instantly; a **URL** becomes a link card
|
||||
(hostname title today; rich OG preview is a backlog item).
|
||||
- **Upload** (toolbar `↑` / empty-state CTA) → the image appears immediately from a blob, then
|
||||
uploads in the background to capture a durable media token (so it becomes reference-capable).
|
||||
- **From library** → the shared gallery picker (Artcraft's own `GalleryModal`), so muscle memory
|
||||
transfers from the rest of the app. Not signed in? The picker now shows a **Log in** button
|
||||
(not a dead "Retry"), routed per-platform via the adapter.
|
||||
- **Note / Color** → a sticky note or an OS-picked swatch, one click each.
|
||||
|
||||
*Rationale:* friction is the enemy of collection. The empty state is an **editorial hero**
|
||||
("Start collecting ideas") that doubles as the drop affordance — it teaches the gestures while
|
||||
looking like a finished product, not a blank slate.
|
||||
|
||||
### 4.2 Organize — keep a growing board legible
|
||||
- **Density** — a segmented control (Compact / Cozy / Comfortable) re-flows the masonry; the
|
||||
layout is target-width-driven, so it adapts to any container (Savee's adjustable-density bar).
|
||||
- **Search + tag chips** — text query over captions/tags/links; click a chip to filter.
|
||||
- **Multi-select** — click, shift/⌘-click, or marquee-drag on bare canvas; a floating glass
|
||||
**selection bar** rises from the bottom with the count + actions.
|
||||
|
||||
### 4.3 Triage — sift keepers from a batch, fast
|
||||
Lightroom is the gold standard for high-volume culling, so we borrow its muscle memory:
|
||||
|
||||
- With items selected, **`1`–`5`** rate, **`0`** clears, **`P`** picks (5 stars). A rating badge
|
||||
persists on the tile.
|
||||
- The toolbar **star filter** cycles "All → 1+ … → 5+ → All" to winnow to keepers.
|
||||
- The selection bar surfaces the shortcuts inline (`1–5 rate · 0 clear`) so they're *discoverable*,
|
||||
not hidden lore.
|
||||
- **Delete is deliberate** — `Delete`/`Backspace` only (we removed `x` as a delete alias: it sits
|
||||
in the number row and the board has no undo, so a stray press must not destroy work).
|
||||
|
||||
*Rationale:* triage is keyboard-first because it's repetitive and high-volume; every key saves a
|
||||
trip to the mouse. Destructive actions get friction; constructive ones get none.
|
||||
|
||||
### 4.4 The board → generate loop — the differentiator
|
||||
This is the move no generic moodboard can make.
|
||||
|
||||
- **"Use as reference"** — on a card's hover rail, in the selection bar, or in the inspector.
|
||||
Selected board images are pushed into the image-generation prompt as style references (via their
|
||||
media tokens) and the user lands on the Create-Image surface, refs pre-filled. Krea/Firefly
|
||||
proved board→generate; Artcraft closes it natively.
|
||||
- The seam is the **adapter** (§6): desktop seeds `usePromptImageStore` + switches to the Image
|
||||
tab; web seeds `useCreateImageStore` + navigates to `/create-image`. Same gesture, same result,
|
||||
two platforms.
|
||||
|
||||
*Rationale:* the highest-leverage feature is the one that turns a passive collection into momentum.
|
||||
It's always one click from any view, and it never blocks — it sends and gets out of the way.
|
||||
|
||||
### 4.5 Inspect — the lightbox as a workbench
|
||||
Double-click or the rail's expand icon opens a full-screen **inspector**:
|
||||
|
||||
- Large media on a glass-blurred backdrop (blur is fine here — it's a fixed overlay), a
|
||||
double-bezel side panel, `←/→` to walk the board, `Esc` to close, background scroll locked,
|
||||
`role="dialog"`.
|
||||
- The panel is a small workbench: **rating** stars, a **tag editor**, an **extracted palette**
|
||||
(client-side color quantization — click a swatch to copy, or "add swatches to board"), and
|
||||
**Use as reference**. The palette is keyed on the image src, so editing tags never re-decodes
|
||||
the image or flickers.
|
||||
|
||||
### 4.6 Canvas — think spatially
|
||||
The freeform Konva stage for arranging, comparing, and planning:
|
||||
|
||||
- Pan/zoom, marquee + lasso select, 8-handle transform, grouping, undo/redo, a shortcut cheatsheet,
|
||||
and auto-layout helpers (pack / fit-to-grid / cluster-by-proximity) that already exist and want
|
||||
better surfacing. Alignment/snapping + a background grid are the next polish (backlog).
|
||||
- Drop a generation onto the canvas, set images side by side, group a coherent set, then send that
|
||||
set to generation. It's the planning room; the grid is the library.
|
||||
|
||||
### 4.7 Present — review & share
|
||||
A distraction-free, full-bleed deck: one item at a time, `←/→/space` to navigate, `Esc` to exit,
|
||||
launched from the view switch as a transient overlay (never persisted, so a reload never reopens
|
||||
into a slideshow). For client/team review and screen-sharing.
|
||||
|
||||
---
|
||||
|
||||
## 5. Micro-UX niceties (the details that read as "expensive")
|
||||
|
||||
- **Hover action rail** — a glass pill on each card fades *up* on hover (translate + opacity), with
|
||||
Open / Use-as-reference / Delete. Buttons stop propagation so they never double as a select.
|
||||
- **Card states choreographed** — hover lifts the shell 2px and deepens the shadow; selected swaps
|
||||
to a 2px brand ring; drag scales to 1.03; drop lands with a snap-overshoot.
|
||||
- **Selection check + rating** — solid scrims (not blur) inside the scrolling grid for legibility
|
||||
without GPU cost.
|
||||
- **Focus-visible rings everywhere** — keyboard users see where they are, not just hover users.
|
||||
- **The Login button** — the gallery picker now invites action ("Log in to browse your library" +
|
||||
a real primary button) instead of a confusing "Retry" link.
|
||||
- **Stagger on entry** — grid items and presentation reveals come in with a short staggered
|
||||
fade-up, so the board feels assembled, not dumped.
|
||||
|
||||
---
|
||||
|
||||
## 6. Cross-platform via the adapter
|
||||
|
||||
One library, two hosts, identical experience. The lib owns everything platform-agnostic; each app
|
||||
injects a `MoodboardAdapter` for its three seams:
|
||||
|
||||
```ts
|
||||
interface MoodboardAdapter {
|
||||
uploadImage?: (file) => Promise<mediaToken | null>; // capture a durable token
|
||||
sendToGeneration: (refs) => void; // board → prompt
|
||||
renderLibraryPicker?: (props) => ReactNode; // the gallery picker (render-prop)
|
||||
}
|
||||
```
|
||||
|
||||
- **Desktop (Tauri):** upload via `@storyteller/api`, picker via `GalleryModal`, send → prompt
|
||||
store + tab switch, login → the login modal.
|
||||
- **Web (artcraft-webapp):** upload via the webapp uploader, picker via the same `GalleryModal`,
|
||||
send → create-image store + `/create-image`, login → navigate `/login`.
|
||||
|
||||
*Rationale:* the adapter is the seam where platform reality lives, so the *experience* stays a
|
||||
single, consistent design across surfaces — and the same polish ships everywhere at once.
|
||||
|
||||
---
|
||||
|
||||
## 7. Motion & performance contract
|
||||
|
||||
- **Curves:** spring (`cubic-bezier(0.32,0.72,0,1)`) for settle/zoom; smooth
|
||||
(`cubic-bezier(0.4,0,0.2,1)`, ~200ms) for hover/selection; snap with slight overshoot on drop.
|
||||
- **Only `transform`/`opacity`/`filter`** animate — never layout properties.
|
||||
- **`backdrop-blur` only on fixed chrome** (toolbar, view switch, inspector/present backdrops);
|
||||
the virtualized scrolling grid uses solid scrims so scrolling stays at 60fps.
|
||||
- **Virtualize aggressively** — deterministic masonry from item aspects means only viewport-visible
|
||||
cards mount; the grid holds thousands of items without lag (the competitor failure mode).
|
||||
- **Respect `prefers-reduced-motion`** — springs become instant, fades remain.
|
||||
|
||||
---
|
||||
|
||||
## 8. Accessibility & polish checklist
|
||||
|
||||
- [x] Theme-correct: consumes `--st-*` only, so light/dark/gray/black/aurora themes work for free.
|
||||
- [x] Focus-visible rings on interactive controls; inspector is a labelled `role="dialog"` with
|
||||
scroll-lock and keyboard nav.
|
||||
- [x] Color is never the only signal (drop state pairs a tint with a dashed outline + icon).
|
||||
- [x] Destructive delete is deliberate (no single-printable-key delete); no silent data loss.
|
||||
- [ ] Cards are not yet keyboard-focusable/selectable (pointer-first) — a known a11y gap.
|
||||
- [ ] Toolbar/inspector don't yet collapse gracefully below ~768px (desktop-first) — responsive
|
||||
pass pending.
|
||||
|
||||
---
|
||||
|
||||
## 9. Where the design goes next
|
||||
|
||||
The research-backed backlog (`FEATURE_BACKLOG.md`) extends this flow without changing its spine —
|
||||
client-side ML (in-browser captioning, auto-tagging, find-similar, palette/color search) makes the
|
||||
board *understand itself*; canvas snapping/alignment and a region→palette tool deepen planning; and
|
||||
the generate-into-a-board / remix loop tightens the differentiator. Collaboration & sharing are
|
||||
deferred behind a flag (they need the backend). The throughline stays constant: **collect with no
|
||||
friction, organize effortlessly, and turn the board into the next generation — calmly.**
|
||||
|
|
@ -1,101 +0,0 @@
|
|||
# Moodboard — UX Change Plan
|
||||
|
||||
> Derived from the recommendations in `DESIGN_AND_FLOW.md` (the high-end design/UX doc), checked
|
||||
> against the **actual implementation**. This is a *design-and-UX polish* plan — closing the gap
|
||||
> between the experience the doc describes and what's shipped. (AI/ML features live separately in
|
||||
> `FEATURE_BACKLOG.md`; backend-dependent work like sharing stays deferred.)
|
||||
|
||||
## Audit — doc says vs. reality
|
||||
|
||||
| Recommendation (DESIGN_AND_FLOW.md) | Status today | Gap |
|
||||
|---|---|---|
|
||||
| §5 "Stagger on entry — items fade-up assembled" | **Not built** — grid renders the virtualized slice with no entry motion | Yes |
|
||||
| §8 "Focus-visible rings on interactive controls" | **Partial** — only card action-rail + toolbar add-buttons | Yes |
|
||||
| §8 "Cards not yet keyboard-focusable/selectable" | **Not built** — pointer-only | Yes (acknowledged) |
|
||||
| §8 "Toolbar/inspector don't collapse < ~768px" | **Not built** — fixed horizontal island; inspector is a fixed 2-pane | Yes (acknowledged) |
|
||||
| §4.2 "Sections" | Model-ready, **no UI** | Yes |
|
||||
| §4.1 "Paste URL → link card" | Hostname-only, **no OG preview** | Yes (needs proxy) |
|
||||
| §4.6 "Auto-layout helpers want better surfacing" | Helpers exist (`pack`/`fitToGrid`/`cluster`), **under-exposed** | Yes |
|
||||
| §5 double-bezel, glass islands, spring motion, hover rail, card states, scrims, login button, triage hint, inspector dialog | **Built** ✓ | — |
|
||||
|
||||
Everything in the bottom row is genuinely shipped; the rows above are the work.
|
||||
|
||||
---
|
||||
|
||||
## The plan (prioritized by impact ÷ effort)
|
||||
|
||||
Each item: **what + UX**, **why** (design rationale), **effort (S/M/L)**, **acceptance**.
|
||||
|
||||
### P1 — Motion & accessibility foundation (do first)
|
||||
|
||||
**1.1 Entry motion — staggered fade-up on the grid.** *(the doc claims it; it isn't there)*
|
||||
- **What:** as cards mount into the viewport, fade-up from `translate-y-2 opacity-0 blur-[2px]` → rest over ~320ms on `--mb-ease-smooth`, with an index-based delay capped at ~12 items so a full screen feels *assembled*, not dumped. Must be virtualization-safe: animate only on first mount, key the animation so re-renders/scroll don't re-trigger; respect `prefers-reduced-motion`.
|
||||
- **Why:** the single biggest "expensive feel" lever the doc names; the board currently pops in flatly.
|
||||
- **Effort:** **S–M.** **Acceptance:** opening a board / switching density plays one clean stagger; scrolling does **not** re-animate; reduced-motion shows instant.
|
||||
|
||||
**1.2 Keyboard-navigable cards.**
|
||||
- **What:** card root becomes focusable (`tabIndex=0`, `role="button"`, `aria-label`); `Enter`/`Space` opens the inspector, `X`/`Delete` deletes the focused card (with the same deliberate-delete rule), and arrow keys move focus across the masonry (geometry-aware: left/right within a row, up/down to the nearest card). Selection follows focus with `Space`; `Shift`+arrows extends.
|
||||
- **Why:** the doc's headline a11y gap; also a power-user speed win (no mouse round-trips during triage).
|
||||
- **Effort:** **M.** **Acceptance:** a keyboard-only user can tab into the grid, move with arrows, open/rate/delete, and select — with a visible focus ring on each card.
|
||||
|
||||
**1.3 Complete focus-visible coverage.**
|
||||
- **What:** add focus-visible rings to every remaining interactive control — `SelectionBar` buttons, `SmartSearchBar` input + clear + tag chips, density segmented control, rating-filter toggle, the view switch, and all `ItemInspector` controls (rating stars, tag input, palette swatches, actions).
|
||||
- **Why:** §8 says "everywhere"; today it's two components.
|
||||
- **Effort:** **S.** **Acceptance:** tabbing through the whole moodboard always shows where focus is.
|
||||
|
||||
### P2 — Responsive & micro-UX completeness
|
||||
|
||||
**2.1 Responsive toolbar.**
|
||||
- **What:** below `md`, the floating island stops being one wide row — it becomes a compact glass bar: board name + a `+`-menu (collapsing the 4 add-buttons), density, rating, and a search **icon** that expands into a sheet; the whole thing `overflow-x-auto` as a last resort so it never breaks the layout.
|
||||
- **Why:** §8 gap; the webapp can be viewed narrow, and today the island overflows.
|
||||
- **Effort:** **M.** **Acceptance:** at 375px wide the toolbar is usable and doesn't clip; at ≥`md` it's unchanged.
|
||||
|
||||
**2.2 Responsive inspector.**
|
||||
- **What:** below `md`, the inspector stacks (media on top, the panel as a bottom sheet) instead of the fixed media-left / 320px-panel split; `h-[100dvh]`; nav arrows move to the corners.
|
||||
- **Why:** §8 gap; the current side-by-side is unusable on phones.
|
||||
- **Effort:** **S–M.** **Acceptance:** inspector is fully usable at 375px.
|
||||
|
||||
**2.3 Sections UI.**
|
||||
- **What:** group items into named, collapsible sections in the grid (the model already supports `sections` + `sectionId`). A section header row with a name (inline-rename), count, collapse toggle, and "new section"; drag a selection into a section; an "Ungrouped" default lane.
|
||||
- **Why:** §4.2 calls it a near-term add; big boards spiral without it (Pinterest Sections / MJ folder-groups).
|
||||
- **Effort:** **M.** **Acceptance:** create/rename/collapse sections; assign items; persists.
|
||||
|
||||
**2.4 Surface canvas auto-layout helpers.**
|
||||
- **What:** a one-click **Tidy / Arrange** menu on the canvas toolbar exposing the already-built `pack` (justified rows), `fitToGrid`, and `cluster-by-proximity`, plus "align selection."
|
||||
- **Why:** §4.6 — the logic exists but is hidden; cheap, high-delight.
|
||||
- **Effort:** **S.** **Acceptance:** the three layouts run from a visible menu; undoable.
|
||||
|
||||
**2.5 "Button-in-button" primary CTAs.**
|
||||
- **What:** apply the doc's nested trailing-icon pattern (already in the empty-state CTA) to the primary actions — "Use as reference" and the inspector's primary button — with the magnetic hover physics (icon translates diagonally + scales on hover, button `active:scale-[0.98]`).
|
||||
- **Why:** §5 haptic consistency; makes the differentiator action feel premium.
|
||||
- **Effort:** **S.** **Acceptance:** primary CTAs share one polished button anatomy.
|
||||
|
||||
### P3 — Larger canvas & capture polish
|
||||
|
||||
**2.6 Canvas alignment + snapping + background grid.**
|
||||
- **What:** snap a dragged node to other nodes' edges/centers with guide lines (DIY on Konva `dragmove`), plus a toggleable background grid + snap-to-grid.
|
||||
- **Why:** §4.6/§9 — makes the freeform canvas feel professional (PureRef 2.1).
|
||||
- **Effort:** **M.** (also tracked in `FEATURE_BACKLOG.md` 3.1/3.2.) **Acceptance:** dragging shows guides + snaps; grid toggles.
|
||||
|
||||
**2.7 Rich URL preview (link cards).**
|
||||
- **What:** paste a URL → fetch OG title + image for a real preview card (vs hostname today).
|
||||
- **Why:** §4.1 — link cards look unfinished without a thumbnail.
|
||||
- **Effort:** **S (client) / M (proxy).** **Dependency:** OG fetch needs a CORS unfurl proxy → **flag as backend-dependent / defer the image; ship title now.**
|
||||
|
||||
**2.8 Drag-from-library onto the board.**
|
||||
- **What:** let a gallery item be dragged directly onto the grid/canvas (not only picked via the modal), reusing Artcraft's existing gallery-drag event.
|
||||
- **Why:** §4.1 — the lowest-friction capture path for already-owned media.
|
||||
- **Effort:** **M.** **Acceptance:** drag a library thumbnail onto the board → it lands as a tokened item.
|
||||
|
||||
---
|
||||
|
||||
## Sequencing
|
||||
|
||||
- **Sprint 1 (feel + a11y):** 1.1 entry motion, 1.2 keyboard cards, 1.3 focus coverage. *One cohesive "the board feels finished and is fully keyboard-able" release.*
|
||||
- **Sprint 2 (responsive + organize):** 2.1 toolbar, 2.2 inspector, 2.3 sections, 2.5 CTAs.
|
||||
- **Sprint 3 (canvas + capture):** 2.4 surface auto-layout, 2.6 snapping/grid, 2.8 drag-from-library, 2.7 link previews (title now, image when a proxy exists).
|
||||
|
||||
## Recommended first move
|
||||
**Sprint 1** — it's the highest "expensive feel ÷ effort" and closes the doc's two explicit a11y
|
||||
gaps + the one motion claim that isn't yet true. All client-side, no backend, fits the shared lib
|
||||
cleanly, and verifiable headlessly (focus order, reduced-motion, DOM-on-mount).
|
||||
Loading…
Reference in New Issue